Program, information processor, and information processing method

ABSTRACT

There is provided a program, an information processor, and an information processing method that make it possible to obtain position information for sites of a body with higher accuracy. The program causes a computer to implement a correction function of referencing a first output obtained by performing a first process on sensor data acquired by two or more motion sensors attached to two or more sites of a body and a second output obtained by performing a second process on the sensor data, and correcting position information for attachment sites to which the motion sensors are attached.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/045,024 (filed on Oct. 2, 2020), which is a National Stage PatentApplication of PCT International Patent Application No.PCT/JP2019/016150 (filed on Apr. 15, 2019) under 35 U.S.C. § 371, whichclaims priority to Japanese Patent Application No. 2018-079335 (filed onApr. 17, 2018), which are all hereby incorporated by reference in theirentirety.

TECHNICAL FIELD

The present disclosure relates to a program, an information processor,and an information processing method.

BACKGROUND ART

In recent years, motion capture techniques for acquiring body motioninformation have been actively developed. The acquired body motioninformation is used, for example, for improvement in forms in sports, orin applications including VR (Virtual Reality) and AR (Augmentedreality).

To acquire body motion information, for example, a position of a motionsensor attached to a body is calculated from sensor data (sensing data)acquired by the motion sensor by using an inertial navigation system(INS) (for example, PTL 1 below).

CITATION LIST Patent Literature

-   PTL1: International Publication No. WO 2017/217050

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In the inertial navigation system, however, the calculation of positionis accomplished by integrating acceleration included in the sensor dataa plurality of times, and therefore an error in the obtained positioncan become large with time. Accordingly, there has been a demand for atechnique for obtaining, with higher accuracy, information regarding theposition of a site to which the motion sensor is attached.

Means for Solving the Problems

According to the present disclosure, there is provided a program thatcauses a computer to implement a correction function of referencing afirst output obtained by performing a first process on sensor dataacquired by two or more motion sensors attached to two or more sites ofa body and a second output obtained by performing a second process onthe sensor data, and correcting position information for attachmentsites to which the motion sensors are attached.

Furthermore, according to the present disclosure, there is provided aninformation processor including a corrector that references a firstoutput obtained by performing a first process on sensor data acquired bytwo or more motion sensors attached to two or more sites of a body and asecond output obtained by performing a second process on the sensordata, and corrects position information for attachment sites to whichthe motion sensors are attached.

Furthermore, according to the present disclosure, there is provided aninformation processing method including; referencing, by a processor, afirst output obtained by performing a first process on sensor dataacquired by two or more motion sensors attached to two or more sites ofa body and a second output obtained by performing a second process onthe sensor data; and correcting, by the processor, position informationfor attachment sites to which the motion sensors are attached.

Effect of the Invention

As described above, according to the present disclosure, it is possibleto obtain, with higher accuracy, information regarding the positions ofthe sites to which the motion sensors are attached.

It should be noted that the above-described effect is not necessarilylimiting, and any of the effects illustrated in the presentspecification or other effects that may be expected from the presentspecification may be achieved together with or instead of theabove-described effect.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an image diagram that describes an overview of a firstembodiment of the present disclosure.

FIG. 2 is an explanatory diagram that describes an existing motioncapture technique using a forward kinematics calculation.

FIG. 3 is an explanatory diagram that describes the forward kinematicscalculation.

FIG. 4 is an explanatory diagram that describes an approach using aninverse kinematics calculation.

FIG. 5 is an explanatory diagram that describes the inverse kinematicscalculation.

FIG. 6 is an explanatory diagram that describes a first step accordingto the embodiment.

FIG. 7 is an explanatory diagram that describes a second step accordingto the embodiment.

FIG. 8 is a schematic diagram illustrating an overview of a positionestimation process based on an inertial navigation system.

FIG. 9 is a schematic diagram illustrating a time-varying image of aposition error that can occur in the position estimation process basedon the inertial navigation system.

FIG. 10 is a schematic diagram illustrating an overview of a positionestimation process based on a dynamics model.

FIG. 11 is a schematic diagram illustrating a time-varying image of aposition error that can occur in the position estimation process basedon the dynamics model.

FIG. 12 is a schematic diagram illustrating an overview of a positionestimation process according to the embodiment.

FIG. 13 is a schematic diagram illustrating a time-varying image of aposition error that can occur in the position estimation processaccording to the embodiment.

FIG. 14 illustrates an example of a unit configuration of a systemaccording to the embodiment.

FIG. 15 illustrates an example of a functional configuration of thesystem according to the embodiment.

FIG. 16 is an explanatory diagram illustrating an example of acombination of attachment sites and a dynamics model.

FIG. 17 illustrates an example of reliability identification by areliability identifier 550.

FIG. 18 is a flowchart illustrating an operation example of the systemaccording to the embodiment.

FIG. 19 illustrates a functional configuration example of a systemaccording to a first modification example.

FIG. 20 illustrates a functional configuration example of a systemaccording to a second modification example.

FIG. 21 illustrates a functional configuration example of a systemaccording to a third modification example.

FIG. 22 is an explanatory diagram that describes an overview of anon-tracking-type dynamics model.

FIG. 23 is an explanatory diagram that describes an overview of atracking-type dynamics model.

FIG. 24 illustrates a functional configuration example of a systemaccording to a fourth modification example.

FIG. 25 illustrates a unit configuration example of a system accordingto a fifth modification example.

FIG. 26 is an explanatory diagram (Part 1) that describes a first ZUPTtechnique according to a second embodiment of the present disclosure.

FIG. 27 is an explanatory diagram (Part 2) that describes the first ZUPTtechnique according to the embodiment.

FIG. 28 is an explanatory diagram (Part 1) that describes a second ZUPTtechnique according to the embodiment.

FIG. 29 is an explanatory diagram (Part 2) that describes the secondZUPT technique according to the embodiment.

FIG. 30 illustrates an example of a functional configuration of a systemaccording to the embodiment.

FIG. 31 is a flowchart illustrating an operation example of the systemaccording to the embodiment.

FIG. 32 is an explanatory diagram that describes estimation of forefootvelocity according to the embodiment.

FIG. 33 is an explanatory diagram that describes estimation of forefootposition according to the embodiment.

FIG. 34 is an explanatory diagram that describes a ZUPT techniqueaccording to modification example 1 of the embodiment.

FIG. 35 is an explanatory diagram that describes estimation of arelative position according to modification example 1 of the embodiment.

FIG. 36 is a flowchart illustrating an operation example of a systemaccording to modification example 1 of the embodiment.

FIG. 37 illustrates an example of a unit configuration of a sensor unit10 according to modification example 2 of the embodiment.

FIG. 38 is an explanatory diagram that describes a ZUPT techniqueaccording to modification example 2 of the embodiment.

FIG. 39 is an explanatory diagram illustrating a hardware configurationexample.

MODES FOR CARRYING OUT THE INVENTION

Preferred embodiments of the present disclosure will be described indetail below with reference to the accompanying drawings. It is to benoted that, in the present specification and drawings, components thathave substantially the same functional configurations are denoted by thesame reference signs, and redundant descriptions thereof are thusomitted.

In addition, in the present specification and drawings, a plurality ofcomponents having substantially the same functional configurations issometimes distinguished from each other by attaching different alphabetsafter the same reference signs. However, in a case where it isunnecessary in particular to distinguish a plurality of componentshaving substantially the same functional configurations from each other,only the same reference signs are assigned.

It is to be noted that the description will be given in the followingorder.

-   -   <<1. Overview of First Embodiment>>    -   <<2. Principle of Present Technology>>    -   <<3. Configuration Example>>    -   <<4. Operation Example>>    -   <<5. Modification Examples>>    -   <<6. Overview of Second Embodiment>>    -   <<7. Hardware Configuration Example>>    -   <<8. Conclusion>>

1. OVERVIEW OF FIRST EMBODIMENT

For example, skeleton information represented by a skeleton structureindicating the structure of a body is used to visualize informationregarding motions of the body of a human, an animal, or the like. Theskeleton structure includes information regarding sites, and bones whichare line segments connecting the sites. The sites in the skeletonstructure each correspond to, for example, an extremity site, a jointsite, or the like. Further, the bones in the skeleton structure maycorrespond to, for example, human bones; however, the positions and thenumber of the bones may not necessarily be consistent with those in theactual human skeleton.

Site positions in the skeleton information are acquirable by, forexample, attaching markers or motion sensors to respective correspondingsites of the body. For example, there exists a technique in whichmarkers are attached to various sites of the body to acquire positionsof the markers using an external camera or the like, and a technique inwhich motion sensors are attached to sites of the body to acquireposition information of the motion sensors on the basis of sensor dataacquired by the motion sensors.

Compared with the technique using markers attached to the body, thetechnique using motion sensors attached to the body is advantageous inthat there is no need for any external camera, light source, or marker,resulting in portability and easiness of setting up. Further, thetechnique using motion sensors attached to the body is free fromlimitations by an image-capturing range of the camera, thus having anadvantage in its availability even in a wide outdoor space, a spacehaving a complicated shape that tends to generate a blind spot, a narrowand small space, etc.

However, in order to estimate a whole-body skeleton with the techniqueusing motion sensors attached to the body, it is necessary to attach amotion sensor to every joint, which sometimes places a heavy load on theuser. Further, although dedicated suits have also been developed tosuitably attach a large number of motion sensors to various sites on theuser, the dedicated suits are sometimes costly to manufacture.

Accordingly, focusing on the above-described circumstances, the presentinventors have created a first embodiment of the present disclosure. Asystem according to the first embodiment of the present disclosure makesit possible to acquire position information for a larger number of sitesthan the number of attached motion sensors while achieving a reductionin the number of the motion sensors to be attached. It is to be notedthat in the present specification, a motion sensor is a device thatsenses motion of the body and may include an inertial sensor (anacceleration sensor, an angular velocity sensor), a geomagnetic sensor,a barometric sensor, an image sensor, and the like. In the following, anexample in which at least an inertial sensor is attached as a motionsensor to the body will be mainly described.

An overview of the present embodiment will be described with referenceto FIG. 1 . FIG. 1 is an image diagram that describes the overview ofthe present embodiment. In the example illustrated in FIG. 1 , sixsensor units 10A to 10F are attached to six sites of the body of a userU1. The sensor units 10A to 10F include, for example, an inertial sensor(IMU: Inertial Measurement Unit) such as an acceleration sensor thatacquires acceleration (Acceleration) or a gyro sensor (an angularvelocity sensor) that acquires angular velocity (Angular Velocity). Thesensor units 10A to 10F are preferably attached to a joint site (e.g.,waist or head) serving as a reference in the body or to the nearextremities of the body (wrist, ankle, head, etc.). In the exampleillustrated in FIG. 1 , the sensor unit 10A is attached to the waist;the sensor units 10B and 10E are attached to the wrists; the sensorunits 10C and 10D are attached to the ankles; and the sensor unit 10F isattached to the head, of the user U1. It is to be noted that in thefollowing description, the sensor units 10A to 10F may be collectivelyand simply referred to as sensor units 10 in a case where it is notnecessary to distinguish them from each other. Further, in thefollowing, sites of the body to which the sensor units 10 are attachedmay also be referred to as attachment sites. In addition, the number ofthe sensor units 10 and attachment positions (positions of theattachment sites) are not limited to those in the example illustrated inFIG. 1 .

The system according to the present embodiment acquires informationregarding position (Position) and orientation (Orientation) of each ofthe attachment sites on the basis of sensor data acquired by the sensorunits 10. As information (hereinafter also referred to as attachmentsite information) for the attachment sites P101 to P106 to which thesensor units 10A to 10F are attached, FIG. 1 illustrates attachment siteinformation PD100 that includes position information and orientationinformation.

Further, on the basis of the position information and the orientationinformation for the attachment sites P101 to P106 to which the sensorunits 10A to 10F are attached, the system according to the presentembodiment estimates skeleton information including position informationand orientation information for various sites of the skeleton structure.Here, in the present embodiment, the position information and theorientation information are estimated for, among sites of the skeletonstructure, not only the attachment sites to which the sensor units 10are attached but also a site to which none of the sensor units 10 isattached (hereinafter also referred to as a non-attachment site).

In the example of FIG. 1 , skeleton information SD100 is illustratedthat includes position information and orientation information forvarious sites in the skeleton structure. The skeleton information SD100includes information for a non-attachment site SP107 as well asinformation for an attachment site SP101 corresponding to the attachmentsite P101 and an attachment site SP102 corresponding to the attachmentsite P102.

In addition to the information for the sites, the skeleton informationmay also include information (position information, orientationinformation, etc.) for bones. For example, in the example illustrated inFIG. 1 , the skeleton information SD100 may include information for abone SB101. For example, on the basis of position information andorientation information for sites in the skeleton structure, it ispossible to identify information for a bone between the sites.

As described above, according to the present embodiment, information fora non-attachment site to which none of the motion sensors is attached isestimated on the basis of information for the attachment sites to whichthe motion sensors are attached. This makes it possible to reduce thenumber of the motion sensors to be attached while maintaining the numberof sites for which information is to be acquired. In the following, adescription is given of the technical principle according to the presentembodiment for achieving the above-described effects.

2. PRINCIPLE OF PRESENT TECHNOLOGY 2-1. Approach of Present Embodiment

As an existing motion capture technique, there is a technique ofacquiring skeleton information by forward kinematics (FK: ForwardKinematics) calculation. The forward kinematics calculation is a methodof calculating the position of an extremity site on that basis oforientations of joint sites.

FIG. 2 is an explanatory diagram that describes the existing motioncapture technique using the forward kinematics calculation. FIG. 2illustrates skeleton information SD90 represented by a skeletonstructure.

In the skeleton information SD90 illustrated in FIG. 2 , attachmentsites to which sensor units including inertial sensors are attached aremarked with circles. Thus, sites SP91 to SP96 are the attachment sites.Here, in a case where the sensor units include angular velocity sensors,it is possible to identify orientation information (angle information)for the attachment sites by using an attitude measurement technique suchas an AHRS (Attitude and Heading Reference System).

For example, in a case where a body coordinate system is set with thesite SP91 corresponding to the waist of the body as a reference (a pointof origin), it is possible to identify position information for thesites SP92 to SP96 in the body coordinate system by performing theforward kinematics calculation on the basis of orientation informationfor the sites SP91 to SP95. Note that in the following description, thesite used as a reference in the body coordinate system will sometimes bereferred to as a root site.

Here, the forward kinematics calculation will be described withreference to FIG. 3 . FIG. 3 is an explanatory diagram that describesthe forward kinematics calculation. For the sake of simplicity, FIG. 3illustrates an example in which an arm is regarded as a simple linkagewith two degrees of freedom.

In the example illustrated in FIG. 3 , a site SP81 is the root site, andthere is illustrated an xy coordinate system (a plane coordinate system)with the site SP81 as the reference (the point of origin). In theforward kinematics calculation, as described above, the orientationinformation for each site is known. In the example illustrated in FIG. 3, an angle θ₁ formed between a bone SB81 and the x-axis based on theorientation of the site SP81, and an angle θ₂ formed between the boneSB81 and a bone SB82 based on the orientation of a site SP82 are known.Further, in the forward kinematics calculation, the distances betweensites, that is, the lengths of bones are also known. In the exampleillustrated in FIG. 3 , the length l₁ of the bone SB81 and the length l₂of the bone SB82 are known.

According to the forward kinematics calculation, it is possible tocalculate a position (x₁, y₁) of a site SP83 using the known informationas described above. Specifically, the position (x₁, y₁) of the site SP83is expressed by the following equations (1) and (2).

[Math. 1]

x ₁ =l ₁ cos θ₁ +l ₂ cos(θ₁+θ₂)  (1)

y ₁ =l ₁ sin θ₁ +l ₂ sin(θ₁+θ₂)  (2)

While FIG. 3 illustrates an example of a planar and simple mechanism, itis also possible to perform the forward kinematics calculation in asimilar manner even in a case of a three-dimensional and more complexmechanism. That is, it is possible to identify the position informationfor an extremity site by using the orientation information for a jointsite between the root site serving as a reference and the extremitysite, and information regarding the distance between the sites. Inaddition, not only the position information for the extremity sites butalso the position information for a joint site located between the rootsite and an extremity site in the skeleton structure (for example, theposition information for the site SP82 in the example illustrated inFIG. 3 ) is identified by the forward kinematics calculation.

One example of existing motion capture techniques using the forwardkinematics calculation has been described above. However, as has beendescribed with reference to FIGS. 2 and 3 , identifying positioninformation for each site by using the forward kinematics calculationnecessitates orientation information for each joint site. Thus, the useof the forward kinematics calculation is a factor responsible for anincrease in the number of motion sensors to be attached to the body.

Accordingly, in the present embodiment, skeleton information is acquiredby using inverse kinematics (IK: Inverse Kinematics) calculation insteadof the forward kinematics calculation described above. The inversekinematics calculation is a method of calculating the orientation ofeach joint site on the basis of the position of an extremity site.

FIG. 4 is an explanatory diagram that describes an approach using theinverse kinematics calculation. FIG. 4 illustrates skeleton informationSD90 represented by a skeleton structure similar to FIG. 2 .

In the skeleton information SD90 illustrated in FIG. 4 , attachmentsites to which sensor units including inertial sensors are attached aremarked with circles. Thus, sites SP91 to SP96 are the attachment sites.Meanwhile, in the skeleton information SD90 illustrated in FIG. 4 , thesites to which any sensor units including inertial sensors areunattached are marked with crosses. Specifically, the sites SP92 toSP95, which are between the site 91 and the site 96, are thenon-attachment sites.

Assume here that a body coordinate system has been set with the siteSP91 corresponding to the waist of the body as the reference (the pointof origin) and the position information for the site SP96 in the bodycoordinate system has been acquired. In such a case, it is possible toidentify orientation information and position information for the sitesSP92 to SP95 by performing the inverse kinematics calculation on thebasis of the position information for the site SP96, which is anextremity site.

Here, the inverse kinematics calculation will be described withreference to FIG. 5 . FIG. 5 is an explanatory diagram that describesthe inverse kinematics calculation. For the sake of simplicity, FIG. 5illustrates an example in which an arm is regarded as a simple linkagewith two degrees of freedom.

In the example illustrated in FIG. 5 , the site SP81 is the root site,and there is illustrated an xy coordinate system (a plane coordinatesystem) with the site SP81 as the reference (the point of origin). Here,as illustrated in the upper left in FIG. 5 , assume that the length l₁of the bone SB81 whose one end point is the site SP81 and the length l₂of the bone SP82 whose one end point is the site SP83 are known.

In the inverse kinematics calculation, an angle formed between the boneSB81 and the x-axis and an angle formed between the bone SB81 and thebone SB82 are calculated using the known information as described above.In the inverse kinematics calculation, however, there can exist aplurality of solutions. In the example of FIG. 5 , obtained are asolution that, as illustrated in the upper right in FIG. 5 , an angleθ₁₁ is formed between the bone SB81 and the x-axis and an angle θ₂₁ isformed between the bone SB81 and the bone SB82; and a solution that, asillustrated in the lower right in FIG. 5 , an angle θ₁₂ is formedbetween the bone SB81 and the x-axis and an angle θ₂₂ is formed betweenthe bone SB81 and the bone SB82. It is to be noted that FIG. 5illustrates an example of a planar and simple mechanism; therefore,there can exist a larger number of solutions in a case of athree-dimensional mechanism or in a case where there are a larger numberof joint sites, for example.

The approach using the inverse kinematics calculation has been describedabove. Next, an overview of processing according to the presentembodiment will be described. In the present embodiment, as describedabove, skeleton information (positions and orientations of joint sites)is acquired using the inverse kinematics calculation. Thus, theprocessing is performed in two general steps as illustrated in FIG. 6and FIG. 7 described below.

FIG. 6 is an explanatory diagram that describes a first step accordingto the present embodiment. As has been described with reference to FIG.1 , in the present embodiment, information regarding acceleration andangular velocity is acquirable from two or more inertial sensorsattached to two or more sites of the body. Further, it is possible toobtain information regarding acceleration (three-dimensional) andorientation (four-dimensional) in a global coordinate system from theinformation regarding acceleration and angular velocity acquired fromthe inertial sensors. It is to be noted that the global coordinatesystem is, for example, a coordinate system for common use by aplurality of sensors, devices, etc., and may be a coordinate systemcorresponding to a real space. The attachment site information PD10illustrated in FIG. 6 includes information regarding acceleration andorientation for the attachment sites P11 to P16 acquired by the inertialsensors, and therefore includes information of 6×7=42 dimensions intotal.

In the first step, position estimation for the attachment sites isperformed (S100) on the basis of the attachment site information PD10,and thereby attachment site information PD12 is obtained that includesposition information indicating positions (three-dimensional) of theattachment sites P11 to P16. It is to be noted that in a case where theattachment site information PD12 is represented in a body coordinatesystem with the attachment site P11 as a root site, the positioninformation for the attachment site P11 is not necessary and thereforethe attachment site information PD12 includes position information of5×3=15 dimensions in total. Further, the attachment site informationPD12 may include the orientation information for each of the sites inthe global coordinate system included in the attachment site informationPD10. In such a case, the attachment site information PD12 includesorientation information of 6×4=24 dimensions in total.

FIG. 7 is an explanatory diagram that describes a second step accordingto the present embodiment. Because the attachment site information PD12illustrated in FIG. 7 is similar to the attachment site information PD12illustrated in FIG. 6 , the description thereof is omitted here.

In the second step, position estimation (interpolation) for anon-attachment site is performed (S200) on the basis of the attachmentsite information PD12, and thereby skeleton information SD10 is obtainedthat includes position information for the non-attachment site to whichno inertial sensor is attached, in addition to the position informationfor the attachment sites. In the example illustrated in FIG. 7 , forexample, sites SP11 and SP12 are the attachment sites and a site SP20 isthe non-attachment site.

Here, the position estimation for the non-attachment site, i.e., thesecond step, may be performed by the inverse kinematics calculation asdescribed above. In order to perform the position estimation for thenon-attachment site with high accuracy, it is desirable that theattachment site information PD12 serving as an input thereto be obtainedwith high accuracy in the first step. The first step, i.e., the positionestimation for the attachment sites, may be performed by an inertialnavigation system, for example. However, in a case where the positionestimation for the attachment sites is performed by the inertialnavigation system, an error in the estimated position can become largerwith time. To cope with this, in the present embodiment, the positioninformation for the attachment sites estimated by the inertialnavigation system is corrected to thereby acquire position informationfor the attachment sites with higher accuracy. The correction ofposition information for the attachment sites according to the presentembodiment is described below.

2-2. Correction of Position Information for Attachment Site

First, a description will be given of a position estimation processbased on the inertial navigation system. The inertial navigation systemis a technique for calculating a sensor position by performingintegration (Integration) of angular velocity and acceleration aplurality of times, and is employed in, for example, vessels, aircraft,or the like. FIG. 8 is a schematic diagram illustrating an overview ofthe position estimation process based on the inertial navigation system.

In the inertial navigation system, first, an orientation of a sensorunit in a global coordinate system is calculated by integrating anangular velocity (an example of sensor data) in a local coordinatesystem acquired by a gyro sensor included in the sensor unit (S11).Next, in the inertial navigation system, on the basis of the orientationof the sensor unit in the global coordinate system, an acceleration (anexample of sensor data) of the sensor unit in the local coordinatesystem (a coordinate system set for each sensor unit) acquired by anacceleration sensor included in the sensor unit is subjected to acoordinate system conversion into an acceleration of the sensor unit inthe global coordinate system (S112). Then, the acceleration of thesensor unit in the global coordinate system resulting from thecoordinate system conversion is integrated (S113) to thereby calculate avelocity of the sensor unit in the global coordinate system. Next, thevelocity of the sensor unit in the global coordinate system isintegrated (S114) to thereby calculate a moving distance of the sensorunit. Here, by combining the moving distances in the global coordinatesystem for each subdivision point, it is possible to obtain relativeposition information with respect to an initial position as a startingpoint. If the initial position is known, it is possible to calculateabsolute position information (i.e., three-dimensional coordinates inthe global coordinate system) of the sensor unit from the aboveinformation. In the above-described manner, orientation information andposition information of the sensor unit are outputted by the positionestimation process based on the inertial navigation system.

The position estimation process based on the inertial navigation systemillustrated in FIG. 8 is performable at a relatively low process loadand a high speed.

Here, in the inertial navigation system, only one integration is appliedto the angular velocity in step S111 in order to obtain the orientationinformation, and further, it is possible to acquire the orientationinformation with higher accuracy by combining a well-known techniquesuch as AHRS. On the other hand, to obtain the position information, twointegrations are applied to the acceleration in step S113 and step S114.Therefore, if the acceleration acquired by the acceleration sensorincludes an error, the error can be accumulated in the outputtedposition information.

FIG. 9 is a schematic diagram illustrating a time-varying image of aposition error that can occur in the position estimation process basedon the inertial navigation system. As illustrated in FIG. 9 , for ashort period of time after the start of estimation of the positioninformation by the inertial navigation system, the position error issmall and it is thus possible to estimate the position information withhigh accuracy. However, the error included in the position informationestimated by the inertial navigation system can increase with time asillustrated in FIG. 9 , and therefore if the estimation is performedcontinuously for a long time, a significantly large error can beincluded in the position information.

To address this, in the present embodiment, position estimation for theattachment sites is performed by regression on the basis of a dynamicsmodel (Dynamics Model). FIG. 10 is a schematic diagram illustrating anoverview of a position estimation process based on the dynamics model.

The process in steps S121 and S122 illustrated in FIG. 10 is similar tothe process in steps S111 and S112 described with reference to FIG. 8 ,and therefore the description thereof is omitted here. As illustrated inFIG. 10 , in the position estimation process based on the dynamicsmodel, position information of the sensor units is estimated byregression (S123) without performing integration of acceleration. In theregression estimation process in step S123, position information isestimated by regression where the orientations and accelerations of thesensor units in the global coordinate system are fitted to a dynamicsmodel prepared in advance. Here, it is possible to generate the dynamicsmodel by, for example, learning kinematic constraint information (forexample, information including a plurality of samples of positions andorientations of various sites in postures or a series of motionsacquired previously) in advance. It is to be noted that variousregression analysis methods are usable for the regression estimationprocess in step S123 and, for example, methods such as DNN (Deep NeuralNetwork) and Random Forest may be used alone or in combination.

Here, because the process illustrated in FIG. 10 involves no integrationof acceleration, a position error is likely to become larger with timeas with the case with the inertial navigation system described withreference to FIGS. 8 and 9 . FIG. 11 is a schematic diagram illustratinga time-varying image of a position error that can occur in the positionestimation process based on the dynamics model. Although a positionerror also occurs in the position estimation process based on thedynamics model as illustrated in FIG. 10 , it does not become largerwith time. Accordingly, a large error is less likely to result even ifthe process is performed continuously for a long time.

Therefore, it is considered that the position estimation process basedon the dynamics model illustrated in FIG. 10 makes it possible toestimate position information with higher accuracy. However, because theposition estimation process based on the dynamics model performsstatistical estimation by regression, variations in output (estimationresult) are likely to be discontinuous even in a case where variationsin input are continuous. As a result, for example, the finally obtainedskeleton information as visualized can tend to give rise to a sense ofstrangeness. Further, when compared with the position estimation processbased on the inertial navigation system illustrated in FIG. 8 , theposition estimation process based on the dynamics model illustrated inFIG. 10 is higher in process load, which makes it difficult to achievehigher execution speed than the position estimation process based on theinertial navigation system.

As described above, the position estimation process based on theinertial navigation system and the position estimation process based onthe dynamics model have their respective features. Therefore, in thepresent embodiment, position information is estimated with higheraccuracy by combining the position estimation process based on theinertial navigation system and the position estimation process based onthe dynamics model, and correcting the estimated position information.In the following, the position estimation process based on the inertialnavigation system may be referred to as a first process, and theposition estimation process based on the dynamics model may be referredto as a second process.

FIG. 12 is a schematic diagram illustrating an overview of a positionestimation process according to the present embodiment. It is to benoted that FIG. 12 illustrates an overview, and thus the positionestimation process according to the present embodiment may furtherinclude a process unillustrated in FIG. 12 .

As illustrated in FIG. 12 , the position estimation process according tothe present embodiment includes a correction process (S130) in additionto the first process (S111 to S114) based on the inertial navigationsystem and the second process (S121 to S123) based on the dynamicsmodel. In the correction process of step S130, correction is performedby referencing an output of the first process (hereinafter, alsoreferred to as a first output) and an output of the second process(hereinafter, also referred to as a second output). It is to be notedthat the first output includes orientation information and positioninformation for the attachment sites, and the second output includesposition information therefor. Then, in step S130, the positioninformation included in the first output is corrected on the basis ofthe orientation information included in the first output and theposition information included in the second output. Note that in stepS130, the orientation information included in the first output may beused for correcting the position information and be outputted as it is.Further, the correction process in step S130 is implementable by aKalman filter, for example.

Moreover, as described above, the first process is executable fasterthan the second process. Therefore, the correction process (S130) may beexecuted at the time when the second output is obtained and, if thesecond output is not obtained but only the first output is obtained, thefirst output may be outputted as it is as an output of the positionestimation process according to the present embodiment.

FIG. 13 is a schematic diagram illustrating a time-varying image of aposition error that can occur in the position estimation processaccording to the present embodiment. Assume that in the exampleillustrated in FIG. 13 , the second output is obtained and correctionbased on the second output is performed at a time t₁₁. As illustrated inFIG. 13 , in the position estimation process according to the presentembodiment, the position error becomes larger with time during a periodfrom the start of the process to the time t₁₁; however, as a result ofthe correction based on the second output performed at the time t₁₁, theposition error is suppressed.

Thus, according to the position estimation process of the presentembodiment, position error is suppressed every time correction isperformed. Therefore, error is less likely to become larger with time,and even if the process is performed continuously for a long period oftime, a large error is less likely to result. Further, in the positionestimation process according to the present embodiment, the first outputis outputted as it is in the case where the second output is notobtainable. This makes it possible to estimate position information morefrequently as compared with a case of performing position estimationwith only the second process based on the dynamics model.

3. CONFIGURATION EXAMPLE

The principle of the technology according to the present embodiment hasbeen described above. Next, a configuration example of the presentembodiment will be described. In the following, a unit configurationexample and a functional configuration example of the present embodimentwill be described in order.

3-1. Unit Configuration Example

FIG. 14 illustrates an example of a unit configuration of a systemaccording to the present embodiment. As illustrated in FIG. 14 , thesystem according to the present embodiment includes the sensor units 10Ato 10F, a hub unit 20, an information processor 30, and an output unit40.

The sensor units 10 are units that each include at least a motion sensorand are to be attached to sites of a body. The sites to which the sensorunits 10A to 10F are attached have been described with reference to FIG.1 , and individual descriptions will thus be omitted. For example, asillustrated in FIG. 14 , the sensor units 10 each include a gyro sensor12, an acceleration sensor 14, a controller 16, and a communicator 18.Note that the motion sensors included in the sensor units 10 are notlimited to inertial sensors (the gyro sensors 12 and the accelerationsensors 14). For example, the sensor units 10 may include motion sensorssuch as geomagnetic sensors, barometric sensors, image sensors or thelike in place of the inertial sensors or in addition to the inertialsensors. Further, while FIG. 14 illustrates the configuration of onlythe sensor unit 10A, the sensor units 10A to 10F may have identicalconfigurations.

The gyro sensor 12 is an inertial sensor that acquires angular velocityas sensor data. The angular velocity acquired by the gyro sensor 12 maybe an angular velocity of the sensor unit 10 in a local coordinatesystem.

Further, the acceleration sensor 14 is an inertial sensor that acquiresacceleration as sensor data. The acceleration acquired by theacceleration sensor 14 may be an acceleration of the sensor unit 10 in alocal coordinate system that is set for each sensor unit 10.

The controller 16 controls operations of the sensor unit 10. Forexample, the controller 16 may control communications performed by thecommunicator 18 and may cause the sensor data (angular velocity, andacceleration) acquired by the gyro sensor 12 and the acceleration sensor14 to be transmitted to the hub unit 20. Alternatively, the controller16 may perform processing on the sensor data acquired by the gyro sensor12 and the acceleration sensor 14, and may cause the processing resultsobtained through the processing to be transmitted to the hub unit 20.

The communicator 18 is a communication module for transmitting andreceiving data to and from other units in a wired manner or wirelessly.The communicator 18 communicates wirelessly with external equipmentdirectly or via a network access point in a scheme such as a wired LAN(Local Area Network), wireless LAN, Wi-Fi (registered trademark)(Wireless Fidelity), infrared communication, Bluetooth (registeredtrademark), short-range/contactless communication or the like, forexample.

The hub unit 20 is an information processor that receives informationfrom a plurality of sensor units 10 and aggregates the information. Itis to be noted that the sensor units 10 and the hub unit 20 may beconnected to each other in a wired manner or wirelessly. The hub unit 20includes, for example, a communicator 22 and a controller 24 asillustrated in FIG. 14 .

The communicator 22 is a communication module for transmitting andreceiving data to and from other units in a wired manner or wirelessly.The communicator 22 communicates wirelessly with external equipmentdirectly or via a network access point in a scheme such as a wired LAN,wireless LAN, Wi-Fi, infrared communication, Bluetooth,short-range/contactless communication or the like, for example.

The controller 24 controls operations of the hub unit 20. For example,the controller 24 may control communications performed by thecommunicator 22 and may cause information received from the sensor units10 to be transmitted as it is to the information processor 30.Alternatively, the controller 24 may perform processing on theinformation received from the sensor units 10, and may cause theprocessing results obtained through the processing to be transmitted tothe information processor 30.

The information processor 30 is an information processor that receivesinformation from the hub unit 20 and processes the information. It is tobe noted that the hub unit 20 and the information processor 30 may beconnected to each other in a wired manner or wirelessly. The informationprocessor 30 includes, for example, a communicator 32 and a controller34 as illustrated in FIG. 14 .

The communicator 32 is a communication module for transmitting andreceiving data to and from other units in a wired manner or wirelessly.The communicator 32 communicates wirelessly with external equipmentdirectly or via a network access point in a scheme such as a wired LAN,wireless LAN, Wi-Fi, infrared communication, Bluetooth,short-range/contactless communication or the like, for example.

The controller 34 controls operations of the information processor 30.For example, the controller 34 performs processing on informationreceived by the communicator 32. Further, the controller 34 is able tocontrol display of the output unit 40 by controlling and causing thecommunicator 32 to transmit the foregoing processing results to theoutput unit 40 connected to the information processor 30. For example,the controller 34 may cause the output unit 40 to display visualizedskeleton information, a 3-D model reflecting the skeleton information orthe like on the basis of the skeleton information obtained as a resultof the processing.

The output unit 40 is a unit that has at least a display function, andperforms displaying in accordance with the control by the controller 34of the information processor 30. The output unit 40 may be, for example,an installation-type display or an HMD (Head Mounted Display) to be wornby a user. Further, the output unit 40 and the information processor 30may be an integral unit.

3-2. Functional Configuration Example

The unit configuration example of the system according to the presentembodiment has been described above. Next, a functional configurationexample of the system according to the present embodiment will bedescribed. FIG. 15 illustrates an example of the functionalconfiguration of the system according to the present embodiment. Notethat FIG. 15 illustrates only the functional configuration that mainlyrepresents the characteristics of the present embodiment, and the systemaccording to the present embodiment may thus include a functionalconfiguration unillustrated in FIG. 15 .

Referring to FIG. 15 , the system according to the present embodimenthas functions as a global motion acceleration orientation calculator510, an integral calculator 520, a selector 530, an attachment positionregression estimator 540, a reliability identifier 550, a ground contactdetector 560, a corrector 570, and an interpolator 580.

It is to be noted that each functional configuration in FIG. 15described below may be achieved by any of the units illustrated in FIG.14 . For example, the controller 16 of the sensor unit 10 or thecontroller 24 of the hub unit 20 may have a function as the globalmotion acceleration orientation calculator 510, and the controller 34 ofthe information processor 30 may have other functions as the integralcalculator 520 to the interpolator 580. However, which unit has whichfunction is not limited to such an example.

The global motion acceleration orientation calculator 510 has a functionof performing processing on the basis of the sensor data (angularvelocities and accelerations) acquired by the sensor units 10A to 10Fillustrated in FIG. 14 . Note that the global motion accelerationorientation calculator 510 may process pieces of the sensor dataacquired respectively by the sensor units 10A to 10F independently ofeach other.

For example, the global motion acceleration orientation calculator 510performs a process of integrating the angular velocities of the sensorunits 10 in the local coordinate system included in the sensor data andthereby calculating orientations of the sensor units 10 in the globalcoordinate system (corresponding to steps S111 and S121 in FIG. 12 ).The global motion acceleration orientation calculator 510 may calculatethe orientations of the sensor units 10 by AHRS. Note that in a casewhere the sensor units 10 include geomagnetic sensors, the global motionacceleration orientation calculator 510 may use sensor data acquired bythe geomagnetic sensors in calculating the orientations of the sensorunits 10 by AHRS.

Further, the global motion acceleration orientation calculator 510performs a process of converting the calculated orientations of thesensor units 10 in the global coordinate system into orientation of theattachment sites to which those sensor units 10 are attached in theglobal coordinate system. The global motion acceleration orientationcalculator 510 may convert the orientations of the sensor units 10 intothe orientations of the attachment sites by using preliminarycalibration results.

Further, on the basis of the orientations of the attachment sites in theglobal coordinate system, the global motion acceleration orientationcalculator 510 performs a process of converting the accelerations in thelocal coordinate system included in the sensor data into accelerationsin the global coordinate system (corresponding to steps S112 and S122 inFIG. 12 ).

Further, the global motion acceleration orientation calculator 510performs a process of removing gravity components from the accelerationsin the global coordinate system obtained as described above. Forexample, the global motion acceleration orientation calculator 510 mayremove the gravity components from the accelerations in the globalcoordinate system on the basis of the orientations of the attachmentsites in the global coordinate system.

The global motion acceleration orientation calculator 510 outputs theorientation information for the attachment sites in the globalcoordinate system obtained by the above-described process and theaccelerations in the global coordinate system, from which the gravitycomponents have been removed, to the integral calculator 520, theselector 530, and the attachment position regression estimator 540.

The integral calculator 520 calculates position information for theattachment sites in the global coordinate system by performing a process(corresponding to steps S113 and S114 in FIG. 12 ) of integrating twicethe accelerations in the global coordinate system, from which thegravity components have been removed, inputted from the global motionacceleration orientation calculator 510. The integral calculator 520outputs to the corrector 570 the calculated position information for theattachment sites in the global coordinate system and the orientationinformation for the attachment sites in the global coordinate systeminputted from the global motion acceleration orientation calculator 510.

Note that the foregoing first process based on the inertial navigationsystem may be performed by the global motion acceleration orientationcalculator 510 and the integral calculator 520 described above. Further,the position information and the orientation information outputted fromthe integral calculator 520 correspond to the first output.

The selector 530 selects a dynamics model for the attachment positionregression estimator 540 to estimate the positions of the attachmentsites by regression. For example, a plurality of dynamics models may beprepared depending on patterns of the sites to which the sensor units 10are attached and may be stored in a memory section (not illustrated).From among such a plurality of dynamics models, the selector 530 selectsa dynamics model to be used by the attachment position regressionestimator 540.

The dynamics models may be prepared in advance for each combination ofthe attachment sites, for example. The selector 530 may select anappropriate dynamics model by identifying the combination of theattachment sites on the basis of information inputted from the globalmotion acceleration orientation calculator 510. Such a configurationallows the selector 530 to dynamically select an appropriate dynamicsmodel even in a case where, for example, one or more of the sensor units10 are detached and re-attached by the user at some point duringoperation of the system or power supply from batteries (not illustrated)is lost in one or more of the sensor units 10. Note that FIG. 15illustrates an example in which the combination of the attachment sitesis identified on the basis of the information inputted from the globalmotion acceleration orientation calculator 510; however, the selector530 may identify the combination of the attachment sites by using thesensor data itself as an input or by using other information.

FIG. 16 is an explanatory diagram illustrating examples of combinationsof the attachment sites and the dynamics models. In each of skeletoninformation SD101 to SD105 illustrated in FIG. 16 , the attachment sitesto which the sensor units 10 are attached are marked with circles. Also,different root sites may be set for different dynamics models.

For example, in the skeleton information SD101 corresponding to awhole-body model, the attachment site SP101 corresponding to the waistis the root site, and the sensor units 10 are attached to six attachmentsites SP101 to SP106. In the skeleton information SD102 corresponding toan upper limb model, the attachment site SP106 corresponding to the headis the root site, and the sensor units 10 are attached to threeattachment sites SP102, P105, and SP106. Further, in the skeletoninformation D103 corresponding to a head-to-arm model, the attachmentsite SP106 corresponding to the head is the root site, and the sensorunits 10 are attached to two attachment sites SP102 and SP106. Further,in the skeleton information D104 corresponding to a waist-to-arm model,the attachment site SP101 corresponding to the waist is the root site,and the sensor units 10 are attached to two attachment sites SP101 andSP105. Further, in the skeleton information D105 corresponding to alower limb model, the attachment site SP101 corresponding to the waistis the root site, and the sensor units 10 are attached to threeattachment sites SP101, SP103, and SP104.

Note that not only the attachment position regression estimator 540 tobe described later but also the corrector 570 and the interpolator 580to be described later perform processes in accordance with thecombination of the attachment sites, and therefore combinations ofnon-attachment sites to be estimated can differ depending on thecombination of the attachment sites (i.e., depending on the dynamicsmodel). Sites for which the system according to the present embodimentis able to estimate position information and orientation information maybe the attachment sites, and non-attachment sites that are locatedbetween a plurality of attachment sites in a predetermined skeletonstructure. In the skeleton information SD101 to SD105 in FIG. 16 ,indicated by hatching are the sites (including attachment sites andnon-attachment sites) for which it is possible to estimate positioninformation and orientation information in the present embodiment, inaccordance with the combinations of the attachment sites.

It should be noted that FIG. 16 illustrates an example, and combinationsof the attachment sites in the dynamics models usable in the presentembodiment are not limited to the example illustrated in FIG. 16 .

Using the dynamics model selected by the selector 530, the attachmentposition regression estimator 540 performs a regression estimationprocess (corresponding to step S123 in FIG. 12 ) of estimating positioninformation for the attachment sites by regression based on theorientation information for the attachment sites in the globalcoordinate system and the accelerations in the global coordinate systemfrom which the gravity components have been removed. Note that theattachment position regression estimator 540 according to the presentembodiment estimates the position information for the attachment sitesin a body coordinate system with reference to the root site set for eachdynamics model. The attachment position regression estimator 540 outputsthe estimated position information for the attachment sites in the bodycoordinate system to the reliability identifier 550 and the corrector570.

Note that the foregoing second process based on the dynamics model maybe performed by the global motion acceleration orientation calculator510 and the attachment position regression estimator 540 describedabove. Further, the position information outputted from the attachmentposition regression estimator 540 corresponds to the second output.

The reliability identifier 550 has a reliability identification functionof identifying reliability of the position information for theattachment sites estimated by the attachment position regressionestimator 540, which will also be referred to as position informationincluded in the second output. The reliability identified by thereliability identifier 550 may be used in corrections by the corrector570 to be described later. For example, in a case where the reliabilityof the position information included in the second output is high, thecorrector 570 to be described later places greater importance on (higherreliability in) the position information included in the second outputthan on the position information included in the first output andperforms a correction: in a case where the foregoing reliability is low,the corrector 570 places greater importance on the position informationincluded in the first output than on the position information includedin the second output and performs a correction.

For example, the reliability identifier 550 may set higher reliabilityin a case where the position information included in the second outputexhibits a narrower likelihood distribution. FIG. 17 illustrates anexample of reliability identification by the reliability identifier 550.In FIG. 17 , a position estimated by the attachment position regressionestimator 540 and its likelihood distribution R10 are illustrated. Forexample, higher reliability is identified in a case where a differencebetween an upper limit L1 and a lower limit L2 of the likelihooddistribution R10 is smaller. In the example illustrated in FIG. 17 ,high reliability is identified, that is, greater importance is placed onthe position information included in the second output, during aninterval between a time t₂₁ and a time t₂₂ and an interval between atime t₂₃ and a time t₂₄. Further, in the example illustrated in FIG. 17, low reliability is identified, that is, greater importance is placedon the position information included in the first output, during aninterval between the time t₂₂ and the time t₂₃.

The reliability identifier 550 outputs the identified reliability to thecorrector 570.

The ground contact detector 560 performs a process of detecting contactof a foot of the body with the ground on the basis of the sensor data(angular velocities and accelerations) acquired by the sensor units 10Ato 10F. Such a configuration makes it possible for the corrector 570described later to convert the position information in the bodycoordinate system estimated by the attachment position regressionestimator 540 into that in the global coordinate system.

The ground contact detector 560 may detect contact with the ground in,for example, a so-called zero velocity update (ZUPT: Zero VelocityUpdate). It is to be noted that ZUPT is a technique of correctingposition by detecting a zero-velocity state as a contact state. In acase where ZUPT is used, the function of correcting the positioninformation by the corrector 570 to be described later is alsoimplementable by ZUPT. Further, methods by which the ground contactdetector 560 detects contact with the ground are not limited to such anexample, and another technique may be used to detect contact of a footof the body with the ground.

The corrector 570 references the first output inputted from the integralcalculator 520 and the second output inputted from the attachmentposition regression estimator 540, and corrects the position informationfor the attachment sites included in the first output. Note that in thefollowing, the function of the corrector 570 may be referred to as acorrection function. By virtue of such a correction function, error isless likely to become larger with time as described with reference toFIGS. 12 and 13 , and it is therefore possible to estimate positioninformation for the attachment sites with higher accuracy.

As described above, the corrector 570 may correct the positioninformation for the attachment sites on the basis of the reliabilityidentified by the reliability identifier 550. Such a configuration makesit possible to estimate the position information for the attachmentsites with higher accuracy.

Further, on the basis of the results of ground contact detection by theground contact detector 560, the corrector 570 converts the positioninformation for the attachment sites in the body coordinate system (thesecond output) estimated by the attachment position regression estimator540 into position information for the attachment sites in the globalcoordinate system. With such a configuration, it is possible to performcorrection with the coordinate systems unified into the globalcoordinate system.

Note that the correction function of the corrector 570 may beimplemented by a Kalman filter. Furthermore, it is possible for theintegration function of the integral calculator 520 described above andthe correction function of the corrector 570 to be implemented incombination by a Kalman filter. For example, it is possible to assume aKalman filter having the velocity and position of each attachment siteas an internal state. Such a Kalman filter integrates the accelerationsof the attachment sites in the global coordinate system outputted fromthe global motion acceleration orientation calculator 510, and therebypredicts the velocities and positions of the attachment sites. Then,such a Kalman filter is able to correct the position information byupdating the internal state using the positions obtained by convertingthe position information included in the second output into that in theglobal coordinate system as an observation.

On the basis of the position information for the attachment sitescorrected by the correcting function of the corrector 570, theinterpolator 580 estimates position information for non-attachment sitesand generates skeleton information. The interpolator 580 may estimatethe position information for a non-attachment site located between aplurality of attachment sites in a predetermined skeleton structure, ashas been described with reference to FIG. 16 . Note that in thefollowing, the function of interpolator 580 may be referred to as aninterpolation function. Such an interpolation function is implementableby, for example, the inverse kinematics calculation described withreference to FIGS. 4 and 5 .

As has been described with reference to FIGS. 4 and 5 , in the inversekinematics calculation, there are typically a plurality of solutions,and there may be a large number of solutions depending on the number ofjoint sites, for example. Therefore, in order to obtain more accurateskeleton information, a mechanism is desired that makes it possible toidentify a more appropriate solution from among a large number ofsolutions present. Thus, the interpolator 580 according to the presentembodiment estimates the position information for the non-attachmentsites by regression with fitting to an inverse kinematics model preparedin advance.

Here, it is possible to generate the inverse kinematics model by, forexample, learning kinematic constraint information (e.g., informationincluding a plurality of samples of positions and orientations ofvarious sites in postures or a series of motions acquired previously) inadvance. It should be noted that various regression analysis methods areusable for the interpolation function of the interpolator 580 and, forexample, methods such as DNN and Random Forest may be used alone or incombination.

Note that although it is described above that the interpolator 580estimates position information for each site, the present embodiment isnot limited to such an example. If either one of the positioninformation and the orientation information for each site is obtained,the other is identifiable in accordance with the skeleton structure.Therefore, the interpolator 580 only has to estimate at least one of theposition information or the orientation information for thenon-attachment sites.

4. OPERATION EXAMPLE

The unit configuration example and the functional configuration exampleof the present embodiment have been described above. Next, an operationexample of the present embodiment will be described. FIG. 18 is aflowchart illustrating an operation example of the system according tothe present embodiment. Note that FIG. 18 illustrates only processesthat mainly represent the characteristics of the present embodiment, andthe system according to the present embodiment may thus execute aprocess unillustrated in FIG. 18 .

As illustrated in FIG. 18 , first, the global motion accelerationorientation calculator 510 calculates the orientations of the sensorunits 10 in the global coordinate system (S502) by AHRS on the basis onthe angular velocities acquired by the gyro sensors 12. Note that stepS502 corresponds to steps S11 and S121 in FIG. 12 .

Subsequently, the global motion acceleration orientation calculator 510converts the orientations of the sensor units 10 in the globalcoordinate system obtained in step S502 into orientations of theattachment sites to which those sensor units 10 are attached in theglobal coordinate system (S504).

Subsequently, the global motion acceleration orientation calculator 510converts the accelerations in the local coordinate system acquired bythe acceleration sensors 14 into acceleration in the global coordinatesystem (S506) on the basis of the orientations of the attachment sitesin the global coordinate system obtained in step S504. Note that stepS506 corresponds to steps S112 and S122 in FIG. 12 .

Subsequently, the global motion acceleration orientation calculator 510removes gravity components from the accelerations in the globalcoordinate system (S508) obtained in step S506 on the basis of theorientations of the attachment sites in the global coordinate systemobtained in step S504.

Subsequently, the integral calculator 520 calculates the positioninformation for the attachment sites in the global coordinate system byintegrating twice the accelerations in the global coordinate system(S510), from which the gravity components are removed, obtained in stepS508. Note that step S510 corresponds to steps S113 and S114 in FIG. 12.

Subsequently, whether or not to perform correction of step S510 isdetermined (S512). As described above, the position estimation processbased on the dynamics model is higher in process load than the positionestimation process based on the inertial navigation system. Accordingly,the execution frequency of the position estimation process based on thedynamics model (S514) is less than that of the position estimationprocess based on the inertial navigation system (S502 to S510). Thus, instep S512, it may be determined that correction is to be performed onceevery time the process illustrated in FIG. 18 is performed apredetermined number of times. However, the present embodiment is notlimited to such an example. For example, in a case where it isdeterminable that the attachment sites are not stationary on the basisof the sensor data, it may be determined that correction is to beperformed, whereas in a case where it is determinable that theattachment sites are stationary, it may be determined that correction isnot to be performed.

In the case where it is determined that correction is to be performed(YES in S512), the attachment position regression estimator 540estimates position information for the attachment sites in the bodycoordinate system by regression estimation using a dynamics model(S514). Note that step S514 corresponds to step S123 of FIG. 12 .

Subsequently, the corrector 570 converts the position information forthe attachment sites in the body coordinate system estimated in stepS514 into position information for the attachment sites in the globalcoordinate system (S516) on the basis of the result of ground contactdetection by the ground contact detector 560.

Then, on the basis of the position information for the attachment sitesin the global coordinate system obtained in step S516, the corrector 570corrects the position information for the attachment sites in the globalcoordinate system (S518) obtained in step S510.

Subsequently, the interpolator 580 estimates (interpolates) positioninformation for the non-attachment sites (S520) by regression withfitting to an inverse kinematics model on the basis of the positioninformation for the attachment sites in the global coordinate system.Note that the position information for the attachment sites used for theregression estimation of the position information for the non-attachmentsites in step S520 may differ depending on the determination result instep S512. In the case where it is determined in step S512 thatcorrection is to be performed, the position information for theattachment sites corrected in step S518 is used for the regressionestimation in step S520. Meanwhile, in the case where it is determinedin step S512 that correction is not to be performed, the positioninformation for the attachment sites in the global coordinate systemobtained in step S510 is used for the regression estimation in stepS520.

The operation of the system according to the present embodiment has beendescribed above. It is to be noted that the processing illustrated inFIG. 18 may be repeatedly executed each time sensor data is acquired bythe gyro sensor 12 and the acceleration sensor 14, or at a predeterminedfrequency.

5. MODIFICATION EXAMPLES

One embodiment of the present disclosure has been described above. Inthe following, some modification examples of the present embodiment willbe described. Note that the modification examples described below may beapplied to the present embodiment singly or in combination. Further,each of the modification example may be applied in place of or inaddition to the configuration described in the present embodiment.

5-1. First Modification Example

For the foregoing embodiment, an example has been described in which theattachment position regression estimator 540 performs the regressionestimation process on the basis of the orientation information for theattachment sites in the global coordinate system calculated by theglobal motion acceleration orientation calculator 510 and theaccelerations in the global coordinate system from which the gravitycomponents have been removed. However, the present technology is notlimited to such an example. For example, angular velocities andaccelerations acquired by the gyro sensors 12 and the accelerationsensors 14 may be used as inputs to such a regression estimationprocess. Such an example will be described as a first modificationexample.

FIG. 19 illustrates a functional configuration example of a systemaccording to the first modification example. Of the configurationillustrated in FIG. 19 , components substantially the same as thoseillustrated in FIG. 15 are denoted by the same reference signs, anddescription thereof will thus be omitted here. A description will begiven below of points different from the system illustrated in FIG. 15 .Referring to FIG. 19 , the system according to the present modificationexample differs from the system illustrated in FIG. 15 in that itincludes an attachment position regression estimator 541 in place of theattachment position regression estimator 540.

The attachment position regression estimator 541 according to thepresent modification example performs the regression estimation processusing angular velocities and accelerations acquired by the gyro sensors12 and the acceleration sensors 14 as inputs, in addition to or in placeof the information calculated by the global motion accelerationorientation calculator 510. A dynamics model to be used for suchregression estimation may be prepared in advance by learning kinematicconstraint information including the same kind of information (angularvelocity and acceleration) as the information used for the regressionestimation.

The first modification example has been described above. According tothe present modification example, using more information makes itpossible to estimate the position information for the attachment siteswith higher accuracy. Note that the above description deals with anexample in which the angular velocity and acceleration are used asinputs to the regression estimation process by the attachment positionregression estimator 541; however, other information may further be usedas inputs. For example, in a case where the sensor units 10 includegeomagnetic sensors or barometric sensors in addition to the gyrosensors 12 and the acceleration sensors 14, information aboutgeomagnetism or barometric pressure may be used as inputs to such aregression estimation process.

5-2. Second Modification Example

Next, an example in which a corrector identifies reliability will bedescribed as a second modification example. FIG. 20 illustrates afunctional configuration example of a system according to the presentmodification example. Of the configuration illustrated in FIG. 20 ,components substantially the same as those illustrated in FIG. 15 aredenoted by the same reference signs, and description thereof will thusbe omitted here. A description will be given below of points differentfrom the system illustrated in FIG. 15 . Referring to FIG. 20 , thesystem according to the present modification example differs from thesystem illustrated in FIG. 15 in that it includes a corrector 572 inplace of the reliability identifier 550 and the corrector 570.

The corrector 572 has, in addition to the function of the corrector 570described with reference to FIG. 15 , a function of identifyingreliability of either the position information for the attachment sitesinputted from the integral calculator 520 or that inputted from theattachment position regression estimator 540, or both. For example, sucha function may be implemented by a Kalman filter, may be implemented tobe performed on the basis of a rule determined in advance, or may beimplemented by regression estimation using a model prepared in advanceby learning.

5-3. Third Modification Example

Next, an example in which a correction process is performed after theposition estimation (interpolation) process for the non-attachment siteswill be described as a third modification example. FIG. 21 illustrates afunctional configuration example of a system according to the presentmodification example. Of the configuration illustrated in FIG. 21 ,components substantially the same as those illustrated in FIG. 15 aredenoted by the same reference signs, and description thereof will thusbe omitted here. A description will be given below of points differentfrom the system illustrated in FIG. 15 . Referring to FIG. 21 , thesystem according to the present modification example differs from thesystem illustrated in FIG. 15 in that it includes a corrector 573 and aninterpolator 583 in place of the reliability identifier 550, thecorrector 570, and the interpolator 580.

The interpolator 583 according to the present modification exampleestimates either position information or orientation information, orboth, for the non-attachment sites by regression on the basis of theorientation information for the attachment sites in the globalcoordinate system calculated by the global motion accelerationorientation calculator 510 and the position information for theattachment sites estimated by the attachment position regressionestimator 540. Note that the regression estimation by the interpolator583 may be similar to the regression estimation for the non-attachmentsites by the interpolator 580 except that information to be inputted isdifferent. The position information or the orientation information forthe non-attachment sites estimated by the interpolator 583 is outputtedto the corrector 573.

It is to be noted that the second process according to the presentmodification example includes an interpolation process of estimating, bythe foregoing interpolator 583, either position information ororientation information, or both, for the non-attachment sites. Then,the second output (the output of the second process) according to thepresent modification example includes either the position information orthe orientation information, or both, for the non-attachment sitesestimated by the interpolator 583. Further, the second output accordingto the present modification example may include the position informationor the orientation information for the attachment sites obtained by theglobal motion acceleration orientation calculator 510 or the attachmentposition regression estimator 540. That is, the second output accordingto the present modification example includes either the positioninformation or the orientation information, or both, for all the sitestargeted for estimation.

On the basis of the first output (the orientation information and theposition information for the attachment sites) inputted from integralcalculator 520, the corrector 573 according to the present modificationexample corrects either the position information or the orientationinformation, or both, for the non-attachment sites included in thesecond output that is inputted from the interpolator 583. In the presentmodification example, the velocities and positions of the attachmentsites and the positions or orientations of the non-attachment sites havea nonlinear relationship, and therefore the correction function of thecorrector 573 according to the present modification example may beimplemented by, for example, an extended Kalman filter. In thefollowing, an example will be described in which a correction functionof correcting the orientation information for the non-attachment sitesis implemented by the extended Kalman filter.

Such an extended Kalman filter has, as an internal state, velocities andpositions of all the attachment sites and orientations of all the sites(including the attachment sites and the non-attachment sites) targetedfor estimation. Such an extended Kalman filter predicts the velocitiesand positions of the attachment sites by integrating the accelerationsof the attachment sites in the global coordinate system outputted fromthe global motion acceleration orientation calculator 510. Then, theextended Kalman filter updates the internal state by using theorientation information for all the sites targeted for estimationincluded in the second output as an observation, thereby being able tocorrect the orientation information. It is to be noted that a covariancematrix of an error that the extended Kalman filter internally has may bepredicted by using a Jacobian that describes the relationship betweenthe velocities of the attachment sites and the orientations of thenon-attachment sites used in the inverse kinematics calculation.

Note that FIG. 21 illustrates the attachment position regressionestimator 540 and the interpolator 583 as separate functionalconfigurations; however, the functions of the attachment positionregression estimator 540 and the interpolator 583 may be unified. Insuch a case, position information and orientation information for allthe sites targeted for estimation may be directly estimated byregression based on the information inputted from the global motionacceleration orientation calculator 510. In such a case, there is aneffect that the number of models to be prepared in advance for theregression estimation is reduced.

5-4. Fourth Modification Example

In the foregoing embodiment, an example in which the attachment positionregression estimator 540 performs regression estimation using anon-tracking-type dynamics model has been described; however, atracking-type dynamics model may be used. In the following, an overviewof the non-tracking-type dynamics model and the tracking-type dynamicsmodel will be described with reference to FIGS. 22 and 23 .

FIG. 22 is an explanatory diagram that describes an overview of thenon-tracking-type dynamics model. As illustrated in FIG. 22 , in theregression estimation using the non-tracking-type dynamics model,information about acceleration and orientation is inputted to allowposition information to be outputted.

FIG. 23 is an explanatory diagram that describes an overview of thetracking-type dynamics model. As illustrated in FIG. 23 , also in theregression estimation using the tracking-type dynamics model,information about acceleration and orientation is inputted to allowposition information to be outputted. However, in the regressionestimation using the tracking-type dynamics model, position informationdelayed by one sample by a delayer D (i.e., the output of a previousregression estimation process) is inputted to a next regressionestimation process in addition to the information about acceleration andorientation. The tracking-type dynamics model has an advantage that thetracking ability is improvable.

An example in which position information for the attachment sites isestimated by regression using such a tracking-type dynamics model willbe described as a fourth embodiment. FIG. 24 illustrates a functionalconfiguration example of a system according to the present modificationexample. Of the configuration illustrated in FIG. 24 , componentssubstantially the same as those illustrated in FIG. 15 are denoted bythe same reference signs, and description thereof will thus be omittedhere. A description will be given below of points different from thesystem illustrated in FIG. 15 . Referring to FIG. 24 , the systemaccording to the present modification example differs from the systemillustrated in FIG. 24 in that it includes an attachment positionregression estimator 544 and a corrector 574 in place of the attachmentposition regression estimator 540 and the corrector 570.

The attachment position regression estimator 544 performs the regressionestimation process by using, as an input, position information for theattachment sites that is corrected by the corrector 574 by usingposition information for the attachment sites estimated in a previousregression estimation process, in addition to the information calculatedby the global motion acceleration orientation calculator 510. Thedynamics model used for such regression estimation is the tracking-typedynamics model described above, and may be prepared in advance bylearning kinematic constraint information including position informationdelayed by one sample, for example.

The fourth modification example described above provides improvedcapability of tracking various motions, thus making it possible toestimate position information for the attachment sites with higheraccuracy.

5-5. Fifth Modification Example

In the foregoing embodiment, a unit configuration example has beendescribed with reference to FIG. 14 ; however, the present technology isnot limited to such an example. In the following, a unit configurationexample of a system that does not include the hub unit 20 will bedescribed as a fifth modification example. FIG. 25 illustrates a unitconfiguration example of a system according to the present modificationexample.

As illustrated in FIG. 25 , the system according to the presentmodification example includes the sensor units 10A to 10F, theinformation processor 30, and the output unit 40. As illustrated in FIG.25 , in the present modification example, the sensor units 10 aredirectly connected to the information processor 30 without the hub unit20 therebetween. It is to be noted that the sensor units 10 and theinformation processor 30 may be connected to each other in a wiredmanner or wirelessly. Except for such a connection relationship, each ofthe units illustrated in FIG. 25 may have a similar configuration tothat of a corresponding unit denoted by the same reference sign in FIG.14 .

5-6. Sixth Modification Example

In the foregoing embodiment, an example in which regression estimationis performed using a dynamics model and an inverse kinematics model hasbeen described. The estimation result of this regression estimation canvary greatly depending on the kinematic constraint information used forlearning to generate the dynamics model and the inverse kinematicsmodel. For example, in a case where kinematic constraint informationobtained by sensing a characteristic posture or motion is used forlearning, the estimation result tends to have a characteristic closer tothe motion used for learning than to the actual motion of the body.

Examples of the characteristic posture or motion include, but are notlimited to, postures unique to characters such as zombies, dynamicmotions typical of fighting scenes, postures frequently appearing ingames such as shooting games, choreographed motions characteristic ofeach genre of dance, motions based on individual rules of each sport,etc.

By appropriately preparing the kinematic constraint information asdescribed above to suit the application and using it for learning togenerate a dynamics model and an inverse kinematics model, it ispossible to estimate skeleton information appropriate to theapplication. As a result, for example, in a case of producing ananimation on the basis of skeleton information, it is possible toproduce a characteristic animation without involving postprocessing onthe skeleton information estimated through the series of processesdescribed above.

6. OVERVIEW OF SECOND EMBODIMENT

In the foregoing embodiment of the present disclosure, as describedpreviously, the process of detecting contact of a foot of the body withthe ground is performed by the ground contact detector 560 in order toconvert the position information in the body coordinate system estimatedby the attachment position regression estimator 540 or the like intothat in the global coordinate system. For example, the ground contactdetector 560 is able to utilize a technique of correcting position byusing ZUPT, in other words, by detecting a state in which the velocityis zero (a predetermined velocity) as a contact state.

6-1. First ZUPT Technique

First, a description will be given of a first ZUPT technique withreference to FIGS. 26 and 27 . FIGS. 26 and 27 are explanatory diagramsthat describe the first ZUPT technique according to the presentembodiment.

The first ZUPT technique utilizes sensor data acquired by the sensorunits 10, which include inertial sensors such as acceleration sensorsand gyro sensors, attached to forefeet (specifically, toes or insteps)as illustrated in FIG. 26 .

Specifically, in the first ZUPT technique, as illustrated in FIG. 27 ,the sensor data is acquired from the foregoing sensor units 10, and theaccelerations of the sensor units 10 in the local coordinate system andthe angular velocities of the sensor units 10 in the local coordinatesystem included in the sensor data are acquired. Then, in the first ZUPTtechnique, the above-described process based on the inertial navigationsystem is performed on the acquired accelerations and angular velocitiesto thereby acquire accelerations, velocities, and positions in theglobal coordinate system (S600).

Next, in this technique, the foregoing sensor data is used to detect astate in which the velocity of a forefoot is zero (S601). In thisdetection, in the technique, the dispersion of the acceleration includedin the foregoing sensor data. the norm of the angular velocity, or thelike may be compared with a threshold value, or the state in which thevelocity is zero may be detected from the acceleration or angularvelocity by machine learning, and thus there is no specific limitation.

Then, in this technique, if it is detected that the velocity of aforefoot is zero, the position of the forefoot in the global coordinatesystem acquired in step S600 by the process based on the inertialnavigation system is corrected (S602) on the assumption that thevelocity of the forefoot in the global coordinate system acquired inS600 by the process based on the inertial navigation system is zero. AKalman filter or the like is usable for the above correction. Forexample, with such a Kalman filter, it is possible to predict theposition of the forefoot in the global coordinate system by integratingthe acceleration of the forefoot in the global coordinate system.Furthermore, the Kalman filter makes it possible to correct the positionof the forefoot in the global coordinate system by updating the internalstate using an observation value where the velocity of the forefoot isassumed as zero.

6-2. Second ZUPT Technique

Next, a second ZUPT technique will be described with reference to FIGS.28 and 29 . FIGS. 28 and 29 are explanatory diagrams that describe thesecond ZUPT technique according to the present embodiment.

Incidentally, in motion capture, the foregoing sensor unit 10 is notalways attached to a forefoot as illustrated in FIG. 26 but may beattached to, for example, an ankle, calf, or the like as illustrated inFIG. 28 . As a characteristic of a human motion, when a forefootcontacts the ground, the velocity of the forefoot surely becomes zerowithin a certain period of time, whereas the velocity of an ankle or thelike is less likely to become zero. In a case where, e.g., a runningmotion is made in motion capture, the above-described tendency maybecome noticeable, in particular. Accordingly, in a case where ZUPT isperformed using sensor data of the sensor unit 10 attached to an ankleor the like, it is sometimes difficult to correct the position in theglobal coordinate system with high accuracy because the velocity is lesslikely to become zero within a certain period of time.

Furthermore, according to the study by the present inventors, in a casewhere a motion of the body obtained by motion capture in a configurationin which the sensor unit 10 is attached to an ankle and no sensor unit10 is attached to a forefoot is reproduced in a skeleton structurerepresenting the structure of the body, accurate reproduction isachievable for the region from the waist to the ankle of a human.However, because a forefoot or a like site places less constraints on ahuman body than a knee or a like site, the motion thereof is difficultto reproduce with high accuracy. In particular, in a case where a bodymotion obtained by motion capture is applied to an animation or the likein which an avatar makes a motion, the avatar's forefeet may sometimesbe unsteady in the animation, resulting in lack of weightiness of theavatar.

In view of the above-described situations, the present inventors haveconceived their own idea of using sensor data of the sensor unit 10attached to an ankle or the like to estimate a state in which thevelocity of a forefoot is zero, and have consequently created the secondZUPT technique according to the present embodiment described below. Thesecond ZUPT technique estimates the velocity of the forefoot indirectly.Therefore, even when a running motion is motion-captured, it is possibleto detect with stability a state in which the velocity of a forefoot iszero, which allows for more robust correction of the position of theforefoot. Furthermore, according to the second ZUPT technique, it ispossible to control the forefeet of an avatar described above by usingthe estimated forefoot velocity or forefoot position. Thus, in a casewhere the motion of the avatar's forefoot contacting the ground isreproduced in an animation, it is possible to reproduce a stableforefoot behavior.

Specifically, in the second ZUPT technique, as illustrated in FIG. 29 ,sensor data is acquired from the sensor unit 10 attached to an ankle,calf, or the like to acquire the acceleration of the sensor unit 10 inthe local coordinate system and the angular velocity of the sensor unit10 in the local coordinate system included in the sensor data. Then, inthe second ZUPT technique, the foregoing process based on the inertialnavigation system is performed on the acquired acceleration and angularvelocity to thereby acquire the acceleration and orientation of thesensor unit 10 in the global coordinate system (S610).

Next, in this technique, the velocity of the forefoot is estimated(S611) by using the sensor data (the acceleration and angular velocityof the sensor unit 10 in the local coordinate system) and theacceleration and orientation of the sensor unit 10 in the globalcoordinate system based on the inertial navigation system. Then, in thistechnique, the position of the forefoot as viewed from the ankle isestimated (S612) by using the sensor data (the acceleration and angularvelocity of the sensor unit 10 in the local coordinate system) and theacceleration and orientation of the sensor unit 10 in the globalcoordinate system based on the inertial navigation system. Furthermore,in this technique, a fusion process of fusing the acceleration andorientation of the sensor unit 10 in the global coordinate system, theestimation result for the velocity of the forefoot, and the estimationresult for the position of the forefoot acquired through the foregoingsteps is performed (S613). As a result, in this technique, it ispossible to acquire corrected positions of the ankle and forefoot in theglobal coordinate system. The details of processing in each step will bedescribed later.

6-3. Functional Configuration Example

The overviews of the first and second ZUPT techniques according to thepresent embodiment have been described above. Next, as one example, asystem functional configuration according to the second ZUPT techniqueof the present embodiment will be described with reference to FIG. 30 .FIG. 30 illustrates an example of the functional configuration of thesystem according to the present embodiment. Note that FIG. 30illustrates only the functional configuration that mainly represents thecharacteristics of the present embodiment, and the system according tothe present embodiment may include the functional configurationillustrated in FIG. 15 .

Referring to FIG. 30 , the system according to the present embodimenthas functions as the global motion acceleration orientation calculator510, the integral calculator 520, the selector 530, the attachmentposition regression estimator 540, the reliability identifier 550, theground contact detector 560, the corrector 570, the interpolator 580,and an estimator 561. It is to be noted that the global motionacceleration orientation calculator 510, the integral calculator 520,the selector 530, the attachment position regression estimator 540, thereliability identifier 550, the ground contact detector 560, thecorrector 570, and the interpolator 580 illustrated in FIG. 30 are thesame as those in the functional configuration of the system according tothe first embodiment described with reference to FIG. 15 . Thus, thedescription thereof will be omitted here and only the estimator 561different from the first embodiment will be described. Further, eachfunctional configuration of FIG. 30 described below may be implementedby any of the units illustrated in FIG. 14 , and is not particularlylimited.

The estimator 561 estimates the velocity of the forefoot and theposition of the forefoot as viewed from the ankle by using the sensordata (the acceleration and angular velocity of the sensor unit 10 in thelocal coordinate system) and the acceleration and orientation of thesensor unit 10 in the global coordinate system from the integralcalculator 520, and outputs the estimation results to the corrector 570.

6-4. Operation Example

The functional configuration example according to the present embodimenthas been described above. Next, an operation example of the second ZUPTtechnique of the present embodiment will be described with reference toFIG. 31 . FIG. 31 is a flowchart illustrating an operation example ofthe system according to the present embodiment. Note that FIG. 31illustrates only the processes that mainly represent the characteristicsof the present embodiment, and the system according to the presentembodiment may execute a process unillustrated in FIG. 31 .

As illustrated in FIG. 31 , the global motion acceleration orientationcalculator 510 and the estimator 561 acquire sensor data (theacceleration and angular velocity of the sensor unit 10 in the localcoordinate system) from the sensor unit 10 attached to the ankle, calf,or the like (S700). Then, the global motion acceleration orientationcalculator 510 and the integral calculator 520 calculate theacceleration and orientation of the sensor unit 10 in the globalcoordinate system (S702) by performing the foregoing process based onthe inertial navigation system on the acceleration and angular velocityacquired in step S700.

Subsequently, the estimator 561 estimates the velocity of the forefoot(S704) by using the sensor data (the acceleration and angular velocityof the sensor unit 10 in the local coordinate system) acquired in stepS700 and the acceleration and orientation of the sensor unit 10 in theglobal coordinate system based on the inertial navigation systemcalculated in step S702. In addition, the estimator 561 estimates theposition of the forefoot (S706) as viewed from the ankle by using thesensor data (the acceleration and angular velocity of the sensor unit 10in the local coordinate system) acquired in step S700 and theacceleration and orientation of the sensor unit 10 in the globalcoordinate system based on the inertial navigation system calculated instep S702.

Next, the corrector 570 executes a correction by the ZUPT process on thevelocity of the forefoot estimated in step S704, and executes a fusionprocess of fusing the position of the forefoot estimated in step S706(S708).

Further, the details of each of steps S704, S706, and S708 illustratedin FIG. 31 will be described with reference to FIGS. 32 and 33 . FIG. 32is an explanatory diagram that describes estimation of forefoot velocityaccording to the present embodiment, and FIG. 33 is an explanatorydiagram that describes estimation of forefoot position according to thepresent embodiment.

(Estimation of Velocity of Forefoot)

The estimator 516 estimates the velocity of the forefoot in step S704described above. Specifically, first, the acceleration and the angularvelocity of the sensor unit 10 in the local coordinate system includedin the sensor data acquired by the sensor unit 10 attached to the ankleare inputted to the estimator 516, as illustrated in FIG. 32 . Further,the acceleration and the orientation of the sensor unit 10 in the globalcoordinate system obtained by performing the above-described processbased on the inertial navigation system on the acceleration and theangular velocity of the sensor unit 10 in the local coordinate systemincluded in the sensor data are inputted to the estimator 516. It is tobe noted that in estimating the velocity of the forefoot, any one or twoof the foregoing sensor data (the acceleration and the angular velocityin the local coordinate system), the foregoing acceleration in theglobal coordinate system, or the foregoing orientation in the globalcoordinate system may be inputted to the estimator 516. Further, inestimating the velocity of the forefoot, sensor data of the sensor units10 attached to other sites, or accelerations and orientations thereof inthe global coordinate system obtained by subjecting that sensor data tothe process based on the inertial navigation system may be inputted tothe estimator 516, and thus there is no specific limitation.

Based on the inputs described above, the estimator 516 then estimatesthe velocity of the forefoot in accordance with pre-described rules. Inthe present embodiment, the estimator 516 may estimate the velocity ofthe forefoot by using machine-learned classifications and regressions.In this case, the estimator 516 is able to output a binary label inwhich the velocity of the forefoot is either zero or non-zero as anestimation result. Further, the present embodiment is not limited tooutputting such a binary label as an estimation result. In the presentembodiment, for example, a ternary or higher-valued label in which thevelocity of the forefoot is “0 m/s² or more and less than 0.01 m/s²”,“0.01 m/s² or more and less than 0.10 m/s²”, “0.10 m/s² or more”, or thelike may be outputted. Further, in the present embodiment, instead ofsuch a label, the velocity itself of the forefoot may be estimated andoutputted.

(Estimation of Position of Forefoot)

The estimator 516 estimates the position of the forefoot as viewed fromthe ankle in step S706 described above. Specifically, as illustrated inFIG. 33 , the acceleration and the angular velocity in the localcoordinate system included in the sensor data acquired by the sensorunit 10 attached to the ankle are inputted to the estimator 516.Further, the acceleration and the orientation of the sensor unit 10 inthe global coordinate system obtained by performing the above-describedprocess based on the inertial navigation system on the acceleration andthe angular velocity of the sensor unit 10 in the local coordinatesystem included in the sensor data are inputted to the estimator 516. Itis to be noted that in estimating the position of the forefoot, one ortwo of the foregoing sensor data (the acceleration and the angularvelocity in the local coordinate system), the foregoing acceleration inthe global coordinate system, or the foregoing orientation in the globalcoordinate system may be inputted to the estimator 516. Further, inestimating the position of the forefoot, sensor data of the sensor units10 attached to other sites, or accelerations and orientations thereof inthe global coordinate system obtained by subjecting that sensor data tothe process based on the inertial navigation system may be inputted tothe estimator 516, and thus there is no specific limitation.Furthermore, body parameters such as height or lengths of leg parts (forexample, leg length above knees or leg length below knees) may beinputted to the estimator 516 as illustrated in FIG. 33 .

Then, on the basis of the foregoing inputs, the estimator 516 estimatesthe position of the forefoot in accordance with pre-described rules. Inthe present embodiment, the estimator 516 may estimate the velocity ofthe forefoot by using machine-learned classifications and regressions.

(Fusion Process)

In step S708 described above, the corrector 570 performs a fusionprocess of fusing the acceleration and the orientation in the globalcoordinate system, the estimated velocity of the forefoot, and theestimated position of the forefoot. The corrector 570 is able to performthe fusion process by using, for example, a Kalman filter or the likethat fuses a plurality of data, placing higher weight on data havinghigher reliability. With such a Kalman filter, for example, in a case ofa binary label in which the velocity of the forefoot estimated in stepS704 is either zero or non-zero, it is possible to correct theobservation value for the label of the zero velocity. Further, with theKalman filter, for example, in a case of a multivalued label havingthree or more values for the estimated forefoot velocity, it is possibleto suitably perform the fusion process by imparting reliability in amanner in which the reliability is sequentially made lower for a labelcloser to the zero velocity to thereby reduce the reliability of thevector of the zero velocity.

As described above, according to the second ZUPT technique of thepresent embodiment, it is possible to estimate the velocity of theforefoot indirectly. Therefore, even when a running motion ismotion-captured, it is possible to detect with stability a state inwhich the velocity of the forefoot is zero, which allows for more robustcorrection of the position of the forefoot. Furthermore, according tothe second ZUPT technique, it is possible to control the foregoingforefeet of an avatar by using the estimated forefoot velocity orforefoot position. Thus, in a case where the motion of the avatar'sforefoot contacting the ground is reproduced in an animation, it ispossible to reproduce a stable forefoot behavior.

6-5. Modification Example 1

In the foregoing embodiment, sensor data of the sensor unit 10 attachedto one ankle is used; however, in the present embodiment, this isnon-limiting and sensor data of the sensor units 10 attached to bothankles may be used. By doing so, it is possible to perform ZUPT withhigher accuracy. Thus, a modification example that uses sensor data ofthe sensor units 10 attached to both ankles will be described withreference to FIGS. 34 to 36 , as modification example 1 of the presentembodiment. FIG. 34 is an explanatory diagram that describes a ZUPTtechnique according to modification example 1. FIG. 35 is an explanatorydiagram that describes estimation of a relative position according tomodification example 1. Further, FIG. 36 is a flowchart illustrating anoperation example of a system according to modification example 1.

Specifically, in the present modification example 1, sensor data isacquired from the sensor unit 10 attached to the right ankle, the sensorunit 10 attached to the left ankle, and the sensor unit 10 attached toanother site, as illustrated in FIG. 34 . Further, in the presentmodification example 1, acceleration and angular velocity of the rightankle in the local coordinate system, acceleration and angular velocityof the left ankle in the local coordinate system, and acceleration andangular velocity of the other site in the local coordinate system areacquired. Then, in the present modification example 1, the foregoingprocess based on the inertial navigation system is performed on theacquired acceleration and angular velocity of the right ankle, andthereby acceleration and orientation of the right ankle in the globalcoordinate system are acquired (S800).

Furthermore, similarly to the foregoing, in the present modificationexample 1, the above-described process based on the inertial navigationsystem is performed on the acquired acceleration and angular velocity ofthe left ankle, and thereby acceleration and orientation of the leftankle in the global coordinate system are acquired (S802). Furthermore,in the present modification example 1, the foregoing process based onthe inertial navigation system is performed on the acquired accelerationand angular velocity of the other site, and thereby acceleration andorientation of the other site in the global coordinate system areacquired (S804).

Next, in the present modification example, respective velocities of bothforefeet are estimated (S806) by using the accelerations andorientations of the right ankle, the left ankle, and the other site inthe global coordinate system based on the inertial navigation system.Then, in the present modification example, respective positions of bothforefeet are estimated (S808) by using the accelerations andorientations of the right ankle, the left ankle, and the other site inthe global coordinate system based on the inertial navigation system.Further, in the present modification example, relative positions (R20illustrated in FIG. 35 ) of both ankles are estimated (S810) by usingthe accelerations and orientations of the right ankle, the left ankle,and the other site in the global coordinate system based on the inertialnavigation system.

In the present modification example, relative positions of both anklesare newly estimated and this makes it possible to acquire positions ofthe ankles and forefeet in the global coordinate system stronger inrestriction. In the estimation, similarly to the estimation in theforegoing embodiment, it is possible to use machine learning. Further,in estimating the velocities of both ankles, the positions of bothankles, and the relative positions of both ankles, not only theaccelerations and orientations of the right ankle, the left ankle, andother site in the global coordinate system based on the inertialnavigation system but also the accelerations and angular velocities ofthe right ankle, the left ankle, and the other site in the localcoordinate system included in the sensor data may be used, or bodyparameters such as height or lengths of leg parts may be used, and thusthere is no particular limitation.

Further, in the present modification example, a fusion process of fusingthe estimation result of the velocities of both forefeet, the estimationresult of the positions of both forefeet, and the relative positions ofboth ankles is performed (S812). Note that as with the foregoing, it isalso possible to perform processing in the fusion process in the presentmodification example by using a Kalman filter or the like that estimatesthe states of velocities and positions of both ankles and both forefeet.As a result, in the present modification example, it is possible toacquire corrected positions of the ankles and forefeet in the globalcoordinate system.

The operation of modification example 1 described above is summarized asfollows. As illustrated in FIG. 36 , the global motion accelerationorientation calculator 510 and the estimator 561 acquire sensor data(accelerations and angular velocities of both ankles and the other sitein the local coordinate system) from the sensor units 10 attached toboth ankles and so on (S820). Then, the global motion accelerationorientation calculator 510 and the integral calculator 520 perform theforegoing process based on the inertial navigation system on theaccelerations and angular velocities acquired in step S820, and therebycalculate accelerations and orientations of both ankles and the othersite in the global coordinate system (S822).

Subsequently, the estimator 561 estimates velocities of both forefeet(S824) by using the accelerations and orientations of both ankles andthe other site in the global coordinate system based on the inertialnavigation system calculated in step S822. In addition, the estimator561 estimates respective positions of both forefeet as viewed from bothankles by using both ankles and the other site in the global coordinatesystem based on the inertial navigation system calculated in step S822.

Next, the estimator 561 estimates relative positions (R20 illustrated inFIG. 35 ) of both ankles (S828) by using the accelerations andorientations of both ankles and the other site in the global coordinatesystem based on the inertial navigation system. Further, the corrector570 performs correction by the ZUPT process to the velocities of bothforefeet estimated in step S824, and performs the fusion process (S830)of fusing the positions of both forefeet estimated in step S826 and therelative positions of both ankles estimated in step S828.

In the present modification example described above, the relativepositions of both ankles are newly estimated by using the sensor data,etc. of the sensor units 10 attached to both ankles, and therefore it ispossible to acquire positions of the ankles and forefeet in the globalcoordinate system stronger in restriction. As a result, in the presentmodification example, it is possible to perform ZUPT with higheraccuracy.

6-6. Modification Example 2

In the foregoing modification example 1, the relative positions of bothankles are estimated; however, sensor data from a distance sensor formeasuring relative positions may further be used. By doing so, itbecomes possible to perform ZUPT with higher accuracy. Thus, amodification example that uses sensor data from a distance sensor formeasuring relative positions will be described with reference to FIGS.37 and 38 , as modification example 2 of the present embodiment. FIG. 37illustrates an example of a unit configuration of the sensor unit 10according to modification example 2, and FIG. 38 is an explanatorydiagram that describes a ZUPT technique according to modificationexample 2.

As illustrated in FIG. 37 , a system according to the presentmodification example 2 may include a sensor unit 10G. The sensor unit10G includes the gyro sensor 12, the acceleration sensor 14, thecontroller 16, and the communicator 18, for example, like the sensorunit 10 illustrated in FIG. 14 . In addition, the sensor unit 10G mayfurther include a distance sensor 50 that is able to measure a relativedistance between both ankles. The distance sensor 50 may be, forexample, an ultrasonic sensor or the like that is able to measure thedistance by the reflection of ultrasonic waves.

Specifically, in the present modification example 2, as illustrated inFIG. 34 , sensor data is acquired from the sensor unit 10 attached tothe right ankle, the sensor unit 10 attached to the left ankle, and thesensor unit 10 attached to another site. Furthermore, in the presentmodification example 2, sensor data regarding the relative distancebetween both ankles is acquired from the distance sensor 50. Then, inthe present modification example 2, the foregoing process based on theinertial navigation system is performed on the accelerations and angularvelocities acquired for both ankles, and thereby accelerations andorientations of both ankles in the global coordinate system are obtained(S900. S902). Further, in the present modification example 2, theforegoing process based on the inertial navigation system is performedon the acceleration and angular velocity acquired for the other site,and thereby an acceleration and an orientation of the other site in theglobal coordinate system are obtained (S904).

Next, in the present modification example, respective velocities andpositions of both forefeet and further, relative positions of bothankles are estimated (S906, S908, S910) by using the accelerations andorientations of the right ankle, the left ankle, and the other site inthe global coordinate system based on the inertial navigation system,and further, the relative distance between both ankles included in thesensor data.

Further, in the present modification example, a fusion process of fusingthe estimation result of the velocities of both forefeet, the estimationresult of the positions of both forefeet, the relative positions of bothankles, and further, the relative distance between both ankles includedin the sensor data is executed (S912).

As above, the foregoing modification example 2 makes it possible toperform ZUPT with higher accuracy by newly using the sensor data of thedistance sensor that measures relative positions.

7. HARDWARE CONFIGURATION EXAMPLE

The embodiments of the present disclosure have been described above.Finally, with reference to FIG. 39 , a description will be given of ahardware configuration of an information processor according to anembodiment of the present disclosure. FIG. 39 is a block diagramillustrating an example of the hardware configuration of the informationprocessor according to the embodiment of the present disclosure. It isto be noted that the information processor 900 illustrated in FIG. 39 isable to implement, for example, the sensor unit 10, the hub unit 20, andthe information processor 30 illustrated in FIG. 14 . Informationprocessing by the sensor unit 10, the hub unit 20, and the informationprocessor 30 according to the embodiment of the present disclosure isimplemented by cooperation of software and hardware described below.

As illustrated in FIG. 39 , the information processor 900 includes a CPU(Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM(Random Access Memory) 903, and a host bus 904 a. Further, theinformation processor 900 includes a bridge 904, an external bus 904 b,an interface 905, an input unit 906, an output unit 907, a storage unit908, a drive 909, a connection port 911, a communication unit 913, and asensor 915. The information processor 900 may include a processingcircuit such a DSP or an ASIC in place of or in conjunction with the CPU901.

The CPU 901 functions as an arithmetic processing unit and a controlunit, and controls overall operations in the information processor 900in accordance with various programs. Further, the CPU 901 may be amicroprocessor. The ROM 902 stores programs, arithmetic parameters, etc.that the CPU 901 uses. The RAM 903 temporarily stores a program that isused in execution of the CPU 901, parameters that change appropriatelyin execution thereof, and the like. The CPU 901 may form, for example,the controller 16, the controller 24, and the controller 34.

The CPU 901, the ROM 902, and the RAM 903 are connected to each other bythe host bus 904 a including a CPU bus and the like. The host bus 904 ais connected to the external bus 904 b such as a PCI (PeripheralComponent Interconnect/interface) bus via the bridge 904. Note that thehost bus 904 a, the bridge 904, and the external bus 904 b are notnecessarily configured separately, and these functions may be packagedin a single bus.

The input unit 906 is implemented by, for example, a unit through whichthe user inputs information, such as a mouse, a keyboard, a touch panel,a button, a microphone, a switch, and a lever. Further, the input unit906 may be a remote-control unit that uses infrared ray or otherelectromagnetic waves, or may be external connection equipment such as amobile phone or a PDA compatible with operations of the informationprocessor 900. Further, the input unit 906 may include, for example, aninput control circuit or the like that generates an input signal on thebasis of information inputted by the user using the input meansdescribed above and outputs the generated input signal to the CPU 901.It is possible for the user of the information processor 900 to inputvarious data or provide instructions for a processing operation to theinformation processor 900 by operating the input unit 906.

The output unit 907 is formed by a unit that is able to notify the userof acquired information visually or audibly. Examples of such a unitinclude a display unit such as a CRT display unit, a liquid crystaldisplay unit, a plasma display unit, an EL display unit and a lamp, asound output unit such as a speaker and a headphone, a printer unit,etc. The output unit 907 outputs, for example, results obtained throughvarious processes performed by the information processor 900.Specifically, the display visually displays the results obtained throughvarious processes performed by the information processor 900 in avariety of formats, such as text, images, tables, graphs, etc.Meanwhile, the sound output unit converts audio signals includingreproduced sound data, acoustic data or the like into analog signals andoutputs the analog signals audibly.

The storage unit 908 is a data storing unit formed as an example of amemory section of the information processor 900. The storage unit 908 isimplemented by, for example, a magnetic memory section device such as anHDD, a semiconductor memory device, an optical memory device, amagneto-optical memory device or the like. The storage unit 908 mayinclude a storage medium, a recording unit for recording data on thestorage medium, a reading unit for reading data from the storage medium,a deletion unit for deleting data recorded on the storage medium, etc.The storage unit 908 stores a program to be executed by the CPU 901,various data, and various externally acquired data, etc.

The drive 909 is a reader/writer for a storage medium, and isincorporated in or externally attached to the information processor 900.The drive 909 reads information recorded on a removable storage mediummounted thereon, such as a magnetic disk, an optical disk, amagneto-optical disk, or a semiconductor memory, and outputs theinformation to the RAM 903. The drive 909 is also able to writeinformation on the removable storage medium.

The connection port 911 is an interface to be connected to externalequipment, and is a connector to the external equipment that is able totransmit data through, for example, a USB (Universal Serial Bus) or thelike.

The communication unit 913 is, for example, a communication interfaceformed by a communication device or the like for connection to a network920. The communication unit 913 may be, for example, a communicationcard or the like for wired or wireless LAN (Local Area Network), LTE(Long Term Evolution), Bluetooth (registered trademark), or WUSB(Wireless USB). Further, the communication unit 913 may be a router foroptical communication, a router for (A ADSL (Asymmetric DigitalSubscriber Line), modems for various types of communications, or thelike. The communication unit 913 is able to transmit and receive signalsor the like to and from the Internet or other communication equipment inaccordance with predetermined protocols such as TCP/IP, for example. Thecommunication unit 913 may form, for example, the communicator 18, thecommunicator 22, and the communicator 32.

The sensor 915 is a sensor of any type such as an acceleration sensor, agyro sensor, a geomagnetic sensor, an optical sensor, a sound sensor, arange-finding sensor, or a force sensor, for example. The sensor 915acquires information about the state of the information processor 900itself, such as the orientation, moving speed or the like of theinformation processor 900, and information about a surroundingenvironment of the information processor 900, such as brightness ornoise around the information processor 900. Further, the sensor 915 mayinclude a GPS sensor that receives a GPS signal to measure the latitude,longitude, and altitude of the unit. The sensor 915 may form, forexample, the gyro sensor 12 and the acceleration sensor 14.

Note that the network 920 is a wired or wireless transmission path forinformation transmitted from units connected to the network 920. Forexample, the network 920 may include a public network such as theInternet, a telephone network or a satellite communication network,various LANs (Local Area Networks) including Ethernet (registeredtrademark), WAN (Wide Area Network), and the like. Further, the network920 may include a dedicated line network such as an IP-VPN (InternetProtocol-Virtual Private Network).

An example of a hardware configuration that is able to implement thefunctions of the information processor 900 according to the embodimentof the present disclosure has been described above. Each of theabove-described components may be implemented by using a general-purposemember, or may be implemented by hardware specialized for the functionof the component. Accordingly, it is possible to appropriately changethe hardware configuration to be used in accordance with the technicallevel at the time of carrying out the embodiments of the presentdisclosure.

It is to be noted that it is possible to create a computer program forimplementing each of the functions of the information processor 900according to the embodiments of the present disclosure as describedabove and to install the computer program in a PC, etc. Further, it isalso possible to provide a computer-readable recording medium in whichsuch a computer program is stored. The recording medium is, for example,a magnetic disk, an optical disk, a magneto-optical disk, a flashmemory, or the like. In addition, the computer program described abovemay be distributed via a network, for example, without using a recordingmedium.

8. CONCLUSION

As described above, according to the embodiments of the presentdisclosure, it is possible to obtain, with higher accuracy, positioninformation for the sites of the body to which the motion sensors areattached. Further, according to the embodiments of the presentdisclosure, it is also possible to obtain position information andorientation information for the sites to which no motion sensors areattached. This makes it possible to obtain skeleton information using asmaller number of motion sensors.

Preferred embodiments of the present disclosure have been describedabove in detail with reference to the accompanying drawings; however,the technical scope of the present disclosure is not limited to suchexamples. It is apparent that those skilled in the art of the presentdisclosure may conceive various alterations or modifications within thescope of the technical idea described in the claims, and it should beunderstood that such alterations and modifications are also within thetechnical scope of the present disclosure.

For example, in the foregoing embodiments, an example in which aninertial sensor is attached to the body as the motion sensor has beenmainly described: however, the present technology is not limited to suchan example. The motion sensor only has to be a sensor that sensesmovements of a body, and may be a geomagnetic sensor, a barometricsensor, an image sensor, or the like as described above.

In addition, the steps in the foregoing embodiments may not necessarilybe processed on a time-series basis in accordance with the orderdescribed herein as the flowchart. For example, the steps in theprocesses of the foregoing embodiments may be processed in an orderdifferent from the order described as the flowchart, or may be processedin parallel.

In addition, the effects described herein are merely illustrative orexemplary, and are non-limiting. That is, the technology according tothe present disclosure may achieve other effects that are apparent tothose skilled in the art from the description herein, in addition to theeffects described above or in place of the effects described above.

It is to be noted that the following configurations are also encompassedby the technical scope of the present disclosure.

(1)

A program that causes a computer to implement a correction function ofreferencing a first output obtained by performing a first process onsensor data acquired by two or more motion sensors attached to two ormore sites of a body and a second output obtained by performing a secondprocess on the sensor data, and correcting position information forattachment sites to which the motion sensors are attached.

(2)

The program according to (1), in which

-   -   the sensor data includes an acceleration,    -   the first process includes integration of the acceleration, and    -   the first output includes the position information for the        attachment sites obtained by the integration.        (3)

The program according to (1) or (2), in which

-   -   the sensor data includes an acceleration,    -   the second process includes a regression estimation process of        estimating the position information for the attachment sites by        regression, and    -   the regression estimation process is performed on the basis of        the acceleration.        (4)

The program according to (3), in which

-   -   the sensor data further includes an angular velocity, and    -   the regression estimation process is performed further on the        basis of the angular velocity.        (5)

The program according to (4), in which the acceleration, after beingsubjected to a coordinate system conversion on the basis of the angularvelocity, is used as an input to the regression estimation process.

(6)

The program according to (4) or (5), in which orientation informationfor the attachment sites obtained on the basis of the angular velocityis used as an input to the regression estimation process.

(7)

The program according to any one of (4) to (6), in which theacceleration or the angular velocity acquired by the motion sensors isused as an input to the regression estimation process.

(8)

The program according to any one of (3) to (7), in which the regressionestimation process is performed on the basis of position information forthe attachment sites previously estimated by the regression estimationprocess.

(9)

The program according to any one of (1) to (8), further causing thecomputer to implement an interpolation function of estimating, on thebasis of the position information for the attachment sites corrected bythe correction function, position information or orientation informationfor a non-attachment site to which none of the motion sensors isattached.

(10)

The program according to (9), in which the interpolation functionestimates the position information or the orientation information forthe non-attachment site located between a plurality of the attachmentsites in a predetermined skeleton structure.

(11)

The program according to (9) or (10), in which the interpolationfunction estimates the position information or the orientationinformation for the non-attachment site by regression.

(12)

The program according to any one of (9) to (11), further causing thecomputer to implement a reliability identification function ofidentifying reliability that indicates reliability of the positioninformation for the attachment sites included in the first output or thesecond output.

(13)

The program according to (12), in which the correction function correctsthe position information for the attachment sites on the basis of thereliability identified by the reliability identification function.

(14)

The program according to any one of (3) to (8), in which the secondprocess further includes an interpolation process of estimating, on thebasis of the position information for the attachment sites estimatedthrough the regression estimation process, position information ororientation information for a non-attachment site to which none of themotion sensors is attached.

(15)

The program according to (14), in which the correction function furthercorrects the position information or the orientation information for thenon-attachment site included in the second output on the basis of thefirst output.

(16)

The program according to any one of (9) to (15), in which the correctionfunction is implemented by a Kalman filter.

(17)

The program according to (1), in which the correction function detects aground contact state and corrects the position information for theattachment sites on the basis of a detection result.

(18)

The program according to (17), in which the correction functionestimates a velocity and a position of a forefoot using an accelerationand an angular velocity included in the sensor data, and detects theground contact state on the basis of an estimation result, the groundcontact state being a state in which the forefoot has a predeterminedvelocity.

(19)

An information processor including a corrector that references a firstoutput obtained by performing a first process on sensor data acquired bytwo or more motion sensors attached to two or more sites of a body and asecond output obtained by performing a second process on the sensordata, and corrects position information for attachment sites to whichthe motion sensors are attached.

(20)

An information processing method including: referencing, by a processor,a first output obtained by performing a first process on sensor dataacquired by two or more motion sensors attached to two or more sites ofa body and a second output obtained by performing a second process onthe sensor data; and correcting, by the processor, position informationfor attachment sites to which the motion sensors are attached.

REFERENCE SIGNS LIST

-   -   10, 10A, 10B, 10C, 10D, 10E, 10F, 10G: Sensor unit    -   12: Gyro sensor    -   14: Acceleration sensor    -   16: Controller    -   18, 32: Communicator    -   20: Hub unit    -   22: Communicator    -   24, 34: Controller    -   30: Information processor    -   40, 907: Output unit    -   50: Distance sensor    -   510: Global motion acceleration orientation calculator    -   520: Integral calculator    -   530: Selector    -   540, 541, 544: Attachment position regression estimator    -   550: Reliability identifier    -   560: Ground contact detector    -   561: Estimator    -   570, 572, 573, 574: Corrector    -   580, 583: Interpolator    -   900: Information processor    -   901: CPU    -   902: ROM    -   903: RAM    -   904: Bridge    -   904 a: Host bus    -   904 b: External bus    -   905: Interface    -   906: Input unit    -   908: Storage unit    -   909: Drive    -   911: Connection port    -   913: Communication unit    -   915: Sensor    -   920: Network    -   D: Delayer    -   D103, D104, D105, SD10, SD90, SD100, SD101, SD102, SD103, SD104,        SD105: Skeleton structure    -   L1: Upper limit    -   L2: Lower limit    -   P11, P12, P13, P14, P15, P16, P101, P102, P103, P104, P105,        P106, SP101, SP102, SP103, SP105, SP106, SP107: Attachment site    -   PD10, PD12, PD100: Attachment site information    -   R10: Likelihood distribution    -   R20: Relative position    -   S100, S111, S112, S113, S114, S121, S122, S123, S130, S200,        S502, S504, S506, S508, S510, S512, S514, S516, S518, S520,        S600, S601, S602, S610, S611, S612, S613, S700, S702, S704,        S706, S708, S710, S800, S802, S804, S806, S808, S810, S812,        S900, S902, S904, S906, S908, S910, S912: Step    -   S53, S54, S55, S56, S57, S58, S59: Position estimation process    -   SB101, SB81, SB82: Bone    -   SP11, SP12, SP20, SP81, SP82, SP83, SP91, SP92, SP93, SP94,        SP95, SP96: Site    -   U1: User

1. A non-transitory computer-readable medium having embodied thereon aprogram, which when executed by a computer causes the computer toexecute an information processing method, the method comprising:estimating velocity information of a forefoot of a user and positioninformation of the forefoot based on sensor data retrieved from a sensorattached to an ankle or a calf of the user; and correcting the positioninformation of the forefoot based on the estimated velocity informationof the forefoot.
 2. The non-transitory computer-readable mediumaccording to claim 1, wherein the position information of the forefootis estimated from a position as viewed from a portion where the sensoris attached.
 3. The non-transitory computer-readable medium according toclaim 1, wherein the velocity information of the forefoot and theposition information of the forefoot are estimated by an inertialnavigation system from a position as viewed from a portion where thesensor is attached by using acceleration data and angular velocity in aglobal coordinate system retrieved as the sensor data at the portionwhere the sensor is attached.
 4. The non-transitory computer-readablemedium according to claim 3, wherein the method further comprises:fusion processing the acceleration data and angular velocity data at theportion where the sensor is attached, the velocity information of theforefoot, and the position information of the forefoot.
 5. Thenon-transitory computer-readable medium according to claim 4, whereinthe method further comprises: outputting the corrected positioninformation at the portion where the sensor is attached and thecorrected position information of the forefoot via the fusionprocessing.
 6. The non-transitory computer-readable medium according toclaim 4, wherein the fusion processing uses a Kalman filter.
 7. Thenon-transitory computer-readable medium according to claim 1, whereinthe method further comprises: estimating velocity information of bothforefeet, position information of both of the forefeet, and relativedistance between both ankles based on sensor data retrieved from sensorsattached at both of the ankles or both calves.
 8. The non-transitorycomputer-readable medium according to claim 7, wherein the methodfurther comprises: correcting fusion processing of the estimatedvelocity information of both of the forefeet, the estimated positioninformation of both of the forefeet, and the relative distance betweenboth of the ankles.
 9. The non-transitory computer-readable mediumaccording to claim 7, wherein the estimating uses machine learning. 10.The non-transitory computer-readable medium according to claim 8,wherein the relative distance between both of the ankles is retrievedfrom a distance sensor.
 11. The non-transitory computer-readable mediumaccording to claim 1, wherein the method further comprises: correctingand further outputting a binary label of the velocity information of theforefoot as an estimation result.
 12. The non-transitorycomputer-readable medium according to claim 1, wherein the methodfurther comprises: correcting and further outputting a ternary orhigher-valued label of the velocity information of the forefoot as anestimation result.
 13. The non-transitory computer-readable mediumaccording to claim 1, wherein the method further comprises: causing anavatar to make a motion in animation by using corrected velocityinformation of the forefoot and the corrected position information ofthe forefoot.
 14. The non-transitory computer-readable medium accordingto claim 1, wherein the method further comprises: reproducing motion ofan avatar's forefoot by using corrected velocity information of theforefoot and the corrected position information of the forefoot.
 15. Thenon-transitory computer-readable medium according to claim 1, whereinthe method further comprises: regression estimating position informationat a portion where the sensor is attached and other portions wheresensors are attached in regression based on the sensor data retrievedfrom the portion where the sensor is attached and the other portionswhere the sensors are attached, and correcting the position informationat the portion where the sensor is attached and the other portions wherethe sensors are attached based on evaluation of the velocity informationof the forefoot.
 16. An information processor comprising circuitryconfigured to: estimate velocity information of a forefoot of a user andposition information of the forefoot based on sensor data retrieved froma sensor attached to an ankle or a calf of the user; and correct theposition information of the forefoot based on the estimated velocityinformation of the forefoot.
 17. An information processing methodcomprising: estimating velocity information of a forefoot of a user andposition information of the forefoot based on sensor data retrieved froma sensor attached to an ankle or a calf of the user; and correcting theposition information of the forefoot based on the estimated velocityinformation of the forefoot.